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PREFACE 


This volume represents tlie fruits of three years of inv i 
(fro’ July 1, 3922, to July 1, 1925) by the Division of Pf 3 
of Institute of Educational llcsearcli. It attempts tr7 
the essential questions coneerninec the nature and meaning, 
measurement of a mental fact in the sample ease of intelligJ- 
rather of a <lefinod segment thereof. Its coneltisions, in 
as they are warranted, should become the basis of sound pr 
in the construction and calibration of scales for use in mental 
suremeut According to them, the present theory and praetit 
measurement of mental abilities are justified to a remarkable 
gi'ee in certain ri'spi'Cfs, but in others should he almost recrea 

Some of the most imix rtant of these conclusions were read 
only in the last six months of the inquiry and are conserjuvjn 
presented uith less adequate o\idential support than is desirah 
The concept of area of intellect in iiartieular needs more expei 
mentation to make it clear, and still more to demonstrate its soum 
ness and i\orth. 

\Vc liad intended to add a long ehaiiter ^evie^^mg the literature 
oil the topics dealt T\ith in this \olunic, but it seemed more impor- 
t<ml TO exeuifilify and apply the results of our conclusions in a 
concrete serit's of tasks selected ami scaled according to the prin- 
ciples deseribeil ; and there was not time to do both. We hope to 
he able to jmbhsh Mich a re^iew later, and in particular to do jus- 
tice to the notable contribution of Kelley (’230), which deserves 
most ear<*ful study by exeiyone A\ho is concerned with the general 
logic of mental nieasnreinents. 

We had intended also to include full treatment of the method 
of obtaining a groujt of approximateh known forms of distribution 
in resiiect of a mental trait measured in truly equal units, by taking 
the members of an array in that trait A\ho have identical scores iii 
"a second trait correlated with the trait in question. This method 
Wii . abandoned in fa^or of a better one, but nearly a third of our 
time and effort xias spent in exploring its possibilities. The results 
should be made known, both because of their intrinsic interest, and 
because otherwise someone will surely be tempted to do again what 
has already been tlone by ns. The material is, however, highly 
technical and elaborate ; and it seemed best not to include it in 
this volume. 
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neral responsibility for the work rests upon the senior 
'ho planned and directed the various inquiries, organized 
ts, and wrote this book, with the exception of Appendix 
would, however, have been utterly impossible for him to 
rried the work through without the financial assistance of 
*’negie Corporation and the Trustees of Teachers College, 
ithout the loyal cooperation of the staff of the Division of 
ology of the Institute of Educational Research, and many 
uifie workers in all parts of the country. Dr. Bregman col- 
I and organized most of the facts which are used in Chapter 
and Appendix III, and some of those used in Chapter VIII. 
s Cobb devised many of the tasks of levels A, B, C, D, E, and 
and, with the aid of Dr. Murdoch, Dr. Tilton and Miss Robin- 
1 , measured 180 imbeciles of mental age 3 to 5 and 100 of mental 
e 6, Dr. Woodyard has arranged and supervised most of the 
sting and scoring in grades 4 to 9, and has shared in the evalua- 
on of the difficulty of the thousands of tasks which have been 
sed in our experiments. Dr. Iklurdoch made all the tests with the 
ifty feeble-minded at Polk. Mrs. Miner has computed most of the 
correlations. Miss Robinson, Dr. Hunsicker, Dr. Tilton, and Mr. 
Upshall have given expert and painstaking service in testing and 
scoring. 

Dr. Toops and Mrs. Ruger worked up the data which j)rovided 
the first set of tasks graded in difficulty from which the final scale 
eventually developed. Miss Hanson, Mrs. Work and Miss Wilcox 
have had a large share in the arrangement and tabulation of the 
results. 


We are indebted, for most courteous and efficient cooperation, 
to all the psychologists on the staff of Teachers College, to fifty 
members of the American Psychological Association who made 
various ratings for us, to Dr. Raymond Pranzen and Dr. Grace A. 
iTaylor who suiiplied valuable records, to Miss Elizabeth E. Par- 
iell. Inspector of Ungraded Classes, New York City, Mr. George 
Bllelcher, Assistant Superintendent of Schools, Kansas City, Mis- 
souri, Dr. B. H. Nifernecker, Director of the Bureau of Educa- 
tional Research of New York City, Dr. Howard W. Potter, Clinical 
Director of Letchworth Village, Dr. Louise M. Poull, Psychologist 
at the Randall’s Island Institution, Mr. Lionel J. Simmons, Super- 
intendent of the Hebrew Orphan Asylum of New York City, and 


towthe many principals and teach 


ers who have facilitated our ex< 
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CHAPTER I 


The Pbesent Status^ 

Existing instruments for measuring intellect® developed 
from three roots, the interview, the school examination, and 
the ^tests' of sensory acuity, memory, attention, and the 
like, devised during ^e early history of psycholq;^. The 
Stanford Binet, for example, is an improved, systematized 
and standardized interview. The Army Alpha is in part an 
improved school examination and in part an improved bat- 
tery of tests like those used before 1900 by Galton, Ebbing- 
haus, Cattell, Jastrow, and others. 

Existing instruments represent enormous improvements 
over what was available twenty years ago, but three funda- 
mental defects remain. Just what they measure is not 
known; how far it is proper to add, subtract, multiply, 
divide, and compute ratios with the measures obtained is not 
known ; just what the measures obtained signify concerning 
intellect is not known. We may refer to these defects in 
order as ambiguity in content, arbitrariness in units, and 
ambiguity in significance. 

AMBIGUITY IN CONTENT 

we examine any of the best existing instruments, say 
the Stanford Binet, the Army Alpha or the National Intel- 
ligence Test, we find a series of varied tasks. Some concern 
words, some concern numbers, some concern space relations, 
some concern pictures, some concern facts of home life. 
Some seem merely informational ; some are puzzle-like. 
Some concern mental activities which will be entirely famil- 
iar to almost all of the individuals to be tested ; some con- 

1 This chapter is reprinted with some alterations from the Psyohologieal 
Beviev), Vol. 31, pp. 219 to 252. 

2 We shall use 'intellect* and 'intelligence* as synonyms throughout 
book. 
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cern novelties. Some are irrespective of speed; in some 
speed is a large element in success. In particular, as we 
shall see later, the score attained is a composite in variable 
proportions whereby A is rated as more intelligent than B 
— ^first, if he can do certain hard tasks with which B fails, 
second, if he can do a greater number than B can of tasks 
of equal difficulty, and third, if he can do more rapidly than 
B tasks at which both succeed. The only sure statement of 
what abilities the Army Alpha measures is to show the test 
itself and its scoring plan. 

To this it may be retorted that this variety is not really 
an ambiguity, that one of these tests is a representative 
sampling of tasks for intellect, and that the scoring plan is 
one which weights each response according to its importance 
as a symptom of intellect. Unfortunately this is not true. We 
may cherish the hope that these tests approximate to such 
representativeness of sampling and suitability of weights. 
In fact, however, nobody has ever made an inventory of 
tasks, determined the correlation of each with intellect, 
selected an adequate battery of them, and found the proper 
weight to attach to each of these. Such a procedure was 
carried out in part by the Committee responsible for the 
construction of the National Intelligence Test, but limita- 
tions of time and funds restricted it to a very small fraction 
of what would be adequate. If anybody did this wisely, a 
large fraction of his labor would be precisely to find out 
what abilities our best present instruments did measure, 
and how these abilities were related to intellect; or to find 
out what abilities cohstituted intellect, and how these abili- 
ties were measured by our present instruments.® 

One of the main lines of work in the improvement of 
instruments for measuring intellect is then to find out what 
abilities our best present instrmnents do measure. 

< Tke balance of his labor might be expended upon experimentation urith 
tariu that seemed promising as ormptoms, even though we did npt know what 
abilities they required. 
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ABBITRABINESS OP UNITS 

The score obtained by using the instrument to measure 
an intellect is in present practice either a number represent 
ting a summation of credits and penalties or, more rarely, 
a number representing the grade of difficulty of the tasks 
which the person can respond to with some assigned i)er- 
centage of correct responses. Thus in Army Alpha he may 
score by summation from 0 to 212 ; in the first suggestion of 
Binet he could score 5 or 6, or 7, or 8, or 9, according as he 
was able to do correctly all but one of the tasks set as 5-year 
tasks, 6-year tasks, 7-year tasks and so on.* 

In neither case (even supposing the measurement to be 
a perfect representation of the person’s abilities) can the 
numbers be taken at their face value. If A scores 50 on 
Alpha, B, 75, and C, 100, we do not know that the difference 
between A and B in the abilities tested by Alpha is the same 
as the difference between B and C, nor that C has twice as 
much of these abilities as A. If D scores mental age 4, E 
mental age 6, and P mental age 8 by the Binet, we do not 
know that, in the abilities tested by the Binet, F excels E as 
much as E excels D, or that F has one and one third times as 
much of these abilities as E has. The numbers, 1, 2, 3, 4, etc., 
designating the scores made by individuals, do not represent 
a series of amounts of intellect progressing by equal steps. 
The difference in intellect between Army Alpha 10 and 
Army Alpha 20 may indeed conceivably be as great as the 
difference between Alpha 100 and Alpha 150. From Stan- 
ford Binet 40 months to 60 months may be as great a 
difference in intellect as from 140 months to 180 months. 
The value of what is called a difference of 1 on the scale is 
not known, and its value may fluctuate greatly as we move 
along the scale. 

*T1u 8 Buggestiou was, howe-ver, abandoned in favor of a procedure wMcb 
mixea two aorta of meaaure. Tbe procedure is, *'Take for point of departure 
tbe age at wbicii ail teats are passed', and l>eyond this age count aa many fiftba 
of a year as there are tests passed.” [^‘The Development of InteUigenee,* Eng. 
trana. of Kite, 1916, p. 278.] 
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We have then no right to add, subtract, multiply, or 
divide with these scores of A, B, C, D, E, and P in the way 
that we do with their heights or weights. Suppose that A 
scores 100; B, 110; C, 90; and D, 120. We cannot say that 
the average intellect of A and B equals the average intellect 
of C and D. If E changes from 60 to 70, while F changes 
from 70 to 80, we cannot say that they have made equal 
gross gains. 

The numbers designating the scores made by individuals 
are usually not even approximately related to any true zero 
point.® Consequently, even if the scores 1, 2, 3, 4, did 
represent an equal-interval series of amounts or degrees of 
the ability in question, they would properly be treated as 
ic + l, T-f- 2, a; + 3, a;-f-4. The ‘times as^ or ratio judg- 
ment is thus not surely applicable and the relations of the 
scores to anything else are thus undetermined. For ex- 
ample, we cannot say whether the intellect of the average 
twelve-year-old is one and a quarter times that of the aver- 
age six-year-old or twice it, or ten times it. 

The second main problem in improving measurements of 
intellect is thus to attach fuller and more definite meanings 
to these credit summations and difficulty levels, and if pos- 
sible to find their equivalents on absolute scales on which 
zero will represent just not any of the ability in question, 
and 1, 2, 3, 4 and so on will represent amounts increasing 
by a constant difference. 

We have to estimate equivalents of this sort somehow 
before we can make much use of ratings by either credit 
summations or difficulty levels ; before, for example, we can 
conveniently compare individuals or groups, or the changes 
made by individuals or by groups, or the effects of different 
environments. The commonest method at present is to take 
as the equivalent for any score by any instrument, the age 
whose average achievement is that score, and to assume that 

^Attempts have been made to define ^ zero’ or ^just not any’ ability and to 
aseififn acores in relation to zero in the case of knowledge of English words, 
ability to understand sentences, handwriting, drawing, and English composition. 



THE PRESENT STATUS 


d 


the increments in average ability are equal for equal differ- 
ences in age up to some limit such as 192 months, and are 
zero thereafter. This of course is purely hypothetical in 
general and is almost certainly in error for the ages near 
the point where the age change suddenly turns from its 
full amount to zero. The curve of ability in relation to age 
is almost always smooth as in the continuous line of Fig. 



Fjq* la* The probable form of the curve of intellect in relation to age. 



Fig. lb. The form of the curve of intellect in relation to age, if annual gains 
are equal up to some stated age, and are zero thereafter. 

la, but not with a sharp turn as in the dash-line of Fig. lb. 
The competent thinkers who use the method know this and 
are cautious in inferences based upon its application to the 
higher ages ; but they use it rather freely for the lower ages. 
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because some method must be used, because it is easy to 
understand and apply, and because we do not know what 
method is really right. 

It may be objected that equality of units is an unneces* 
sary refinement, for present practical purposes, since the 
mental age defines the status of an individual sufficiently, 
‘as able as the average ten-year-old,’ ‘as able as the average 
twelve-year-old.’ These, it may be said, are better measure- 
ments for practical purposes than some absolute scale in 
terms of equal ‘mentaces’ or ‘intels.’ The convenience, in- 
telligibility and realism of the mental age scale up to about 
12 or 13 years are indeed great advantages, but after 13 or 
14 it is neither convenient nor intelligible nor realistic. It 
is not convenient because the computation of intelligence 
quotients becomes very troublesome for the higher ages. It 
is not readily intelligible because mental ages 14, 15, 16, etc., 
are not ‘as good as the average’ 14-year old, 15-year old, etc. 
The average 25-year old for example is about the mental age 
of 14 by one of the best instruments. It is not realistic 
because we have no clear or vivid sense of what the average 
person is intellectually at fifteen, or at sixteen, and do not 
even know whether he improves in the next two or three 
years. A mental age of 15 or 16 or 17 is in fact as arbitrary 
a quantity as an Alpha ability of 123. 

A rarer but more promising procedure than that of trans- 
forming test scores into ‘ages’ is to transform them into 
units of ability on the assumption that the distribution of 
ability in all adults 21-30, or in all twelve-year-olds, or in all 
pupils in grade six of a certain city, or in some other speci- 
fied group, is approximately that given by 


1 

y — — =c‘*' 

^ 0V2tt 


For example, the Alpha scores from 0 to 212 were not 
used in the army at their face value, which would give a 
distribution of the form shown in Fig. 2, but were trans- 
muted into seven letter measures by the following sch^e, 
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whicli assumed an approximately * normal’ distribution for 
a random sampling of 128,747 of the literate white draft ; 



Fig. 2. The form of distribution of the literate white draft if Army Alpha 

are taken at their face Talue« 

13f>-212 = A 
105-134 = B 
75^104 = C + 

45- 74 = C 
25- 44 = C— 

15- 24 = D 
0- 14 = D — 

The score used in the Thorndike-MeCall test of para- 
graph reading is not the number of correct answers, but a 
transmutation on the assumption that the real ability con- 
cerned is distributed * normally’ amongst twelve-year-olds 
in American cities. 

We know very little concerning the permissibility of the 
assumption of the so-called normal distribution for adults 
or for an age, or for a school grade. The search for evi- 
dence pro and eon is one important feature of the attempt 
to obtain units of mental ability which shall be at least ap- 
proximately equal. 

AM3R||tlllA||||r IN SIGNIFICANCE 

The teCI meiuBures directly only the measurer’s 

impression Ty<W^e,sul|||ct’s performance, or the summa- 
tion in ^ ^i^pricious fashion, of credits and 

penalties responses to the different ele- 

ments of tlnsiiililiift;: the a t^mbination of these. What this 
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score signifies about the subject’s intellect depends upon 
the intuition of the measurer, or upon the correlation be- 
tween the summation and intellect, or upon both. When we 
assert that a child is found by measurement with the Stan- 
ford Binet to have the intellect of a child 10^ years, all that 
is really asserted is that the child does as well in that par- 
ticular standardized interview as did the average of the 
children of 10^ years of age tested by Terman in making 
his standards. We do not know what the average intellect 
of these children was, nor how closely the Stanford Binet 
score represents or parallels or signifies it. 

Wlien we assert that a man is found by measurement 
with the Army Alpha to have the intellect of an average 
recruit in the draft, all that is really asserted is that he does 
as well in that particular battery of tests scored and sum- 
mated in a particular way, as the average recruit did. J ust 
what the intellects of recruits were and how closely their 
Alpha scores paralleled their intellects, we do not know. 
The measurement is one thing, the inference to intellect is a 
different thing. 

This is of course true of many measurements. The 
amount of silver deposited in one second by an electric cur- 
rent is not the amount of current. The dividend rate on 
stock during any one year is not the worth of the stock. The 
amount of silver is, under proper conditions, of perfect sig- 
nificance as an indicator of the amount of current, since the 
correlation between it and a perfect criterion of amount of 
current is perfect. The dividend rate is of very imperfect 
significance, since the correlation between it and a perfect 
criterion of the worth of the stock is far from perfect. 

We do not know how closely the rating or score in the 
Stanford Binet or the Army Alpha or any other instrument 
correlates with a perfect criterion of intellect, because we do 
not know what such a criterion is, much less its correlations 
with these tests. One great task of the measurement of 
intellect is to obtain such a criterion, or a closer approxima- 
tion to it than we now have, and to use it to improve the 
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selection and weighting of the elements of our testing in- 
struments. 

The present status of such instruments as the Binet or 
Army or National tests is roughly as follows: We have 
chosen tests where the judgment of sensible people in gen- 
eral is that correct response or speed of correct response is 
characteristic of intellect. Such is the case with directions 
tests, arithmetical problems, common sense questions (as 
in Alpha 3), and the like. We have chosen tests using the 
judgment of psychologists in the same way. Such is the 
case with the completion tests devised hy Ebbinghaus, the 
mixed relations or analogies test devised by Woodworth, 
and the like. We have tried these or other tests with chil- 
dren secluded in institutions because of imputed intellec- 
tual inferiority and with children of like age who are in 
ordinary schools (as by Norsworthy), with adult males of 
good health and morals who were found in a Salvation 
Army home, glad to work for a dollar a day, and with 
adults of the professional classes (as by Simpson), with 
children in general of different ages (as by Binet and 
Terman), with various groups of children ranked for im- 
puted intelligence by teachers, fellow pupils, school ad- 
vancement, and other symptoms (as by Spearman, Burt, 
Terman, Whipple, Yerkes, and others), with children of 
alleged superior intelligence in comparison with others (as 
by Whipple and Terman), with soldiers in the National 
Guard and regular army in connection with ratings for 
intelligence given by their officers (as by the Psychology 
Committee of the National Research Council), with stu- 
dents whose success in h( 'h school and college studies was 
also measured (as by Coivin, Wood, and many others), 
with individuals who were tested with a very long series 
of tests (as by Terman and Chamberlain, Stenquist, and 
others), and in other ways. 

As a general result we know that certain systematized 
interviews and batteries of tests measure somewhat the 
same trait, since they correlate somewhat one with an- 
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other; and that this trait has to some extent the same coii> 
stitution as the trait which sensible people, psychologists, 
and teachers rate as intellect. 

The failure of perfect correlation between the amount 
of intellect a person has, as revealed by the criterion, and 
the amount indicated by the instrument is due, as has been 
said, partly to the imperfection of the criterion, but partly 
also to the imperfection of the instruments. They (at 
least all but one of them) are demonstrably imperfect, since 
no two of them correspond perfectly in their findings for 
the same intellects. Since it is extremely unlikely that, 
out of a dozen instruments devised with about equal care 
by a dozen individuals or committees at about the same 
date one should be very much superior to all the others, 
we may assume, until there appears proof to the contrary, 
that all are imperfect. 

The imperfection may be of two sorts. First, the re- 
sponses measured by the instrument may not be represen- 
tative of the whole intellect and nothing but intellect; the 
score obtained may not give enough weight to certain fac- 
tors or elements of intellect and may give weight to others 
which really deserve less or even zero weight. The instru- 
ment is then like a wattmeter which gives only half weight 
to the voltage of the current or adds two watts for every 
time that the current is turned on or turned off. Second, 
the same person may receive a different score when re- 
measured by the instrument. In so far as such differences 
are due to the ‘accidental’ ups and downs in the person’s 
achievements, they are taken care of by measuring him at 
enough different times ; but in so far as they are due to ac- 
quaintance with the instrument itself or with instruments 
li&e it, they are a very serious imperfection. For example, 
a given score with Army Alpha represents a very different 
status according as it is from a first, a second, or a third 
trial. The case here is as if a thermometer tended after 
subjection to a temperature of 200“ once to register 220“ 
when 200* was next encountered. The provision of means 
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for distinguishing between that part of the score due to 
certain general characteristics of the person measured and 
that part of the score which is due to certain special train- 
ing that he has had with the tasks of the tests, or with 
tasks like them, is thus an important part of the work of 
making the measurements more fully and exactly signifi- 
cant of intellect. 

In general, all our measurements assume that the indi- 
vidual in question tries as hard as he can to make as high a 
score as possible. None of them can guarantee that the 
scores would correspond at all with a perfect criterion if 
the individuals measured tried to appear as dull as they 
could. The correlation would indeed then probably be in- 
verse, the more intelligent persons being more successful 
in their efforts to appear dull! It is theoretically possible 
to arrange a system of incentives such that each person 
measured by an instrument would put forth approximately 
his maximum effort, and in scientific testing of the instru- 
ments this can often be done. In general practice, how- 
ever, we rarely know the relation of any person’s effort to 
his possible maximum effort. Since, how’^ever, the disturb- 
ances due to differences in effort on the part of those tested 
require in study and treatment procedures which have little 
or nothing to do with the procedures by which the instru- 
ments are made to give better measurements of those who 
do try their best, we shall disregard the former and shall 
limit our inquiry to the latter sort of procedures. 

MEASUREMENTS OF INTELLIGENCE ABE MEASURES OF 
INTELLECTUAL PRODUCTS 

All scientific measurements of intelligence that we have 
at present are measures of some product produced by the 
person or animal in question, or of the way in which some 
product is produced. A is rated as more intelligent than B 
because he produces a better product, essay written, answer 
found, choice made, completion supplied or the like, or pro- 
duces an equally good product in a belter way, more quickly 
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or by inference rather than by rote memory, or by more in- 
genious use of the material at hand. 

We can conceive of states of affairs such that a man’s 
intellect could be measured without consideration of the 
products he produces or the ways in which he produces 
them. Intellect might be exactly proportionate to the ac- 
tivity of the thyroid gland, or to the proportion of the brain 
weight to body weight, or to the number of associative neu- 
rones in the frontal lobes or to the complexity of the fibril- 
lary action of certain neurones, or to the intensity of a 
certain chemical process, and hence be measurable by ob- 
servations of the thyroid’s action, or estimates of the 
brain’s volume, or by a count or measurement of neurones, 
or by a chemical analysis. 

Psychologists would of course assume that differences 
in intelligence are due to differences histological or physio- 
logical, or both, and would expect these physical bases of 
intelligence to be measurable. At present, however, we 
know so little of the neural correlates of intellect that if 
twenty college freshmen were immolated to this inquiry, 
ten being the most intellectual of a hundred, and ten 
being the least intellectual of the hundred, and their 
brains were studied in every way by our best neurologists, 
these could probably not locate sixteen out of the twenty 
correctly as at top or bottom. Moreover, what we do know 
of neural correlates is of little avail during life, the living 
neurones being extremely inaccessible to present methods 
of observation. 

Kven if one aimed at discovering the physiological basis 
of intellect and measuring it in physiological units, one 
would have to begin by measuring the intellectual products 
produced by it. For our only means of discovering physio- 
logical bases is search for the physiological factors which 
correspond to intellectual production. 

MEASUBEMElirTS OF INTEL<L,IGENCE IMPLY VALUATION 

Our present measurements of intelligence rest on human 
judgments of value, judgments that product A is * better’ 
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or ‘truer’ or ‘more correct’ than product B, that method O 
is ‘preferable’ to method D, or that C is ‘right’ while D is 
‘wrong,’ and the like. 

In some cases this ii^ so clear that everyone must admit 
it. Thus in three of our best tests of intelligence, giving 
the opposites of words, completing sentences by supplying 
omitted words, and answering questions about a paragraph 
read, we make elaborate keys assigning credits to the dif- 
ferent responses.® These keys are obviously made by 
human judgments of the value of each response. 

The credits given may represent valuations by the truth- 
fulness or wisdom of the answers or sentences, by their 
grammatical form, by their rhetorical excellence, by their 
originality, by the rate of producing them, or by a subtle 
sense of their significance as evidence of intelligence. 

6 For example, the task being to complete, 

^God made . • , and . • • let Mm pass for a man,’ we find among the 
Tesponsea of high-school graduates: 


him 

therefore 

him 

so 

him 

then 

him 

will 

him 

they 

him 

ho 

him 

I 

him 

let 

man 

always 

man 

then 

man 

God 

man 

has 

man 

he 

man 

therefore 

man 

please 

Adam 

then 

Adam 

Eve 

Ada^k 

he 

f 

, wouldn ’t 

God 

* wo 


OMtll 


and must asBigfn 4 jading line between fnll value 

and no value 
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In some cases the value is assigned so easily (as a simple 
deduction from, or following of, a general rule) that we may 
thoughtlessly assume that the response indicates intelli- 
gence regardless of any process of valuation. For ex- 
ample, we may consider that in a test in arithmetical com- 
putation or problem solving, the right answers are signs of 
intelligence, regardless of what anybody thinks. A little 
thought will convince us, however, that in such tests the 
human judgment acts as truly as in a completion or para- 
graph-reading test. The main difference is that, having 
once for all decided that right answers are better than 
wrong answers, we do not raise the issue about any par- 
ticular answer. We simply assume or make a general rule 
of valuation. The valuation becomes obvious if we col- 
lect all the responses made to an arithmetical task and ask 
whether all the different ‘rights’ are equally good or right, 
and whether all the different ‘wrongs’ are equally undesir- 
able.^ 

One criterion of value, trytli, is so widely used in fram- 
ing, keying, and scoring tests of intelligence that it deserves 
comment, especially since there may be in the case of truth 
an objective criterion, power in prediction, by which our 
judgments of value are or should be determined. Two 
other criteria of value also need comment because they have 
been suggested explicitly or implicitly as direct criteria 
for intelligence. They are developmeivt with age and abil- 
ity to learn. 

TEUTH 

Probably over half of our present tests of intelligence 
are tests where the response is given credit as a symptom 
of intelligence in proportion to its truthfulness. Such is 
the case, for example, with eight out of ten tests of the Otis 

7 In the special case where we arrange for Yes and No answers Talnation is 
doubly active. We arrange so that a Yes or a No will be 'good* as a response. 
Then, since some of the correct ^Yeses’ or *Noes^ may be due to chance, and 
since chance answers are deemed of no value, we plan our scoring so as to give 
the chance 'Yeses’ and 'Noes* zero value. 
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Advanced ; and with Army Alpha, 2, part of 3, 4, 6, 7, and 8. 
It is more or less the case with Stanford Binet III, 5 ; IV, 
1, 2, 3; V, 1, 2, 3, 4; VI, 1, 2, 3, 5; VII, 1, 2, 5; VIII, 4, 5, 6; 
IX, 1, 2, 3; X, 1, 2; XU, 1, 2, 8; XIV, 1, 2, 3, 5, 6; and with 
National Intelligence A, 1, 3, 4, and B, 1, 2, 3, and 5. 

One conld make an attractive theory of intelligence and 
its measurement somewhat as follows: Intellect is con- 
cerned with facts, being the ability to see and learn the 
truth, to get true knowledge and use it to the beft advan- 
tage. Truth is insight into the real world, the evidence 
that knowledge is true is its predictive power. Measures 
of intelligence are then ultimately measures of a man’s 
mastery of prediction, that 2 and 2 will be 4, or that it will 
be profitable to buy such and such a stock, or that a planet 
will be found having such and such a path. More immedi- 
ately, thej’’ are measures of certain abilities which contrib- 
ute to, or accompany, or indicate the existence of, the abil- 
ity to get and use the truth. 

By this theory we should rest our valuations of truth 
all on the ultimate test of power of prediction. One truth 
would be better than another in proportion as it predicted 
more facts, or more important facts, or predicted the same 
facts more acccurately, or helped more in the acquisition of 
other truths. Our valuations of abilities as evidences of 
intellect would rest on their significance as symptoms of 
ability to get and use truth. 

It seems sure, however, that people in general, psychol- 
ogists, and framers of intelligence tests, alike mean by 
intellect something more than ability in truth-getting to 
improve prediction. They mean what Pericles and Wash- 
ington and Gladstone had as well as what Aristotle and 
Pasteur and Darwin had. In the oral interview of the 
business man or physician to test intelligence, in such tests 
as Ebbinghaus’ completions, and in such a battery of tests 
as Army Beta, there is little obvious reference to predic- 
tion or truth getting. In the first case, the aim is rather 
to see how the person fits his thoughts and acts to little 
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problems or emergencies; in the second, it was rather to 
give him a chance to use all the so-called higher mental 
powers; in the third, many tasks were selected in which 
people who were regarded as intelligent could do better 
than people regarded as dull, and those of them which most 
conveniently distinguished the alleged bright from the al- 
leged dull were kept as the final choice. If these instru- 
ments do really measure ability at truth getting, it is only 
indirectly and more by accident than by design. 

It may be that truth-getting is what we unwittingly do 
measure by our intelligence tests, or what we ought to try 
to measure, but very few of those who devise or apply the 
tests think so. And it is surely wise to find out what we 
do measure before deciding that it is or ought to be truth- 
getting. 

DEVBLOnVtENT WITH AGE 

Binet had it in mind to discover those intellectual abili- 
ties which six-year-olds had that five-year-olds did not have, 
those which seven-year-olds had that five-year-olds and 
six-year-olds did not have, and so on. It might seem that, 
except for the one judgment that abilities were ‘better^ or 
represented * greater intelligence^ the later they came in 
this series of normal chronological process, the Binet mea- 
surement would be free from valuation. 

However, valuation came in from the start because Binet 
tried only abilities which he valued as intellectual. He did 
not take all the psychological features of five-, six-, and 
seven-year-olds and choose as his series of tests those which 
separated the ages most distinctly. In revising Binet ’s 
series Terman and others have paid less and less attention 
to lateness of development and more and more to signifi- 
cance as valued symptoms of intelligence in their choice of 
tasks. 

This is well. For if Binet or they had collected a series 
of tasks such as showed the least overlapping of one chrono- 
logical age on the next, the resulting series would be in- 
ferior as a measure of intellect to the series as it stands. 
For example, quality of handwriting, rate of tapping, and 
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ability in checking A’s on a mixed sheet of capitals would 
probably show less overlapping with age than vocabnlary, 
rate of reading, and ability in completing sentences. But 
they would be far less ^effective in diagnosing amount of 
intelligence. 

Development with age would be a poor and partial crite- 
rion for intellect of any sort or degree, and for the higher 
ranges of intellect, say those above the 70-percentile intel- 
lect of the average of the white draft, or above the average 
ninth-grade pupil, it would be well-nigh worthless. It has 
never been so used. The Terman mental ages above 14, 
for example, are not functions of development with age, but 
of differences between individuals, regardless of age. 

ABmiTY TO LEARN 

All obvious hypothesis, often advanced, is that intellect 
is the ability to learn, and that our estimates of it are or 
should be estimates of ability to learn. To be able to learn 
harder things or to be able to learn the same thing more 
quickly would then be the single basis of valuation. Suc- 
cess in solving arithmetical problems, or defining words, or 
completing sentences would then be good, simply and solely 
because it signified that the person had greater ability to 
learn. 

If greater ability to learn means in part ability to learn 
harder things, we have excluded the vague general valua- 
tion of certain products and ways of producing only to in- 
clude it again. For we shall find ourselves selecting or 
defining A as harder to learn than B on the ground that 
only the more intellectual persons can do it, or on the 
ground that it requires a higher type of intellect, and shall 
find ourselves using those vague general valuations to pick 
the persons or describe the type of intellect required. 

If greater ability to learn means onlv the ability to learn 
more things or to learn the same things more quickly, we 
have a view that has certain advantages of clearness and 
approximate fitness to many facts. Even less than in the 
case of truth-getting, however, do our present actual instru- 
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ments for measuring intelligence measure directly a per- 
son's ability to learn more things than another person can, 
or to learn the same things more quickly. The substitution 
test included in Army Beta, in the National Intelligence 
Examination and in some others, is about the only test of 
speed of learning that is used ; and it is more than a learn- 
ing test. 

Much evidence will therefore be required before we can 
wisely replace our present multifarious empirical valua- 
tions by the formula that intellect is the ability to learn 
more things or to learn the same things more quickly. 

The reduction of all valuations of response to valuation 
as symptoms of ability to learn more and more quickly 
thus seems too narrow a view. It has other defects. Were 
it true, we ought, other things being equal, to get better 
correlations with a criterion of intellect from tests in learn- 
ing something new and from tests deliberately framed to 
measure how much one has learned in life so far, than from 
the existing batteries of miscellaneous tasks. 

This does not seem to be the case. Quantitative data 
concerning individual differences in learning under experi- 
mental conditions are rather scanty, and their correlations 
with a criterion of intellect are scantier still ; but what facts 
we have been able to gather do not show that, per hour of 
time spent, tests in learning predict the criterion as well 
as do the tests now in use. Tests framed to measure how 
much one has learned in life so far, such as vocabulary 
tests, information tests, or such Binet elements as ‘Knows 
whether he is a boy or a girl,' and the like, are valuable, 
but not, so far as we can determine, more valuable than a 
composite containing also tests primarily of selective, rela- 
tional, generalizing, and organizing abilities. 

OTHER ATTEMPTED SIMPLIFICATIONS OF THE PROCESS 

OF VALUATION 

Response to Novelty 

In one way allied to the doctrine just described and in 
one way sharply contrasted with it, is the doctrine that a 
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person’s intellect is measured by his ability to respond well 
to new situations, to do ‘originals.’ The importance of 
some such ability as this will, of course, be admitted. How- 
ever, in view of the great difficulty of deciding just what 
situations are ‘new’ for any given individual; in view of 
the fact that ‘to respond well’ is likely to bring in many 
or all of our vague general valuations again ; in view of the 
fact that distinctions among novel situations as ‘harder’ 
(that is, making greater demands on intellect) will have the 
same effect ; and in view of the fact that our most approved 
present instruments include many tasks which seem as 
fittingly called responses to the familiar as to the new — 
in view of all this it seems best at present not to try to 
narrow our valuations to fit this theory. 

Relational Thinking 

Spearman has argued that intellect equals the appre- 
hension of experience, the eduction of relations and the 
eduction of correlates. The two processes are defined as 
follows: “The mentally presenting of any two or more 
characters (simple or complex) tends to evoke immediately 
a knowing of relation between them.” [23, p. 63.] “The 
presenting of any character together with any relation 
tends to evoke immediately a knowing of the correlative 
character.” [23, p. 91.] 

There is no doubt that the appreciation and manage- 
ment of relations is a very important feature of intellect, 
by any reasonable definition thereof. Yet it seems hazard- 
ous and undesirable to assume that the perception and use 
of relations is all of intellect. In practice, tests in para- 
graph reading, in information, in range of vocabulary, 
seem to signify intellect almost flweU as tliPl^llposites and 
mixed relations tests. In theor;C (tldin|^ng things 

into their elements), selection 8iboaB|^ table ele- 
ments or aspects or relations), jrad managing 

many associative trends so thaj P|j ||8l^ ici weight 

in view of the purpose of one^P |^ i|aj||h^7^pi^ to be as 
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deserving of consideration as the perception and use of 
relations. Moreover, I fear that, in all four cases, we need 
other valuations to decide which are the better relations 
or more abstract relations, or the more essential elements, 
or the more sagacious selection, or the more consistent 
organization, or the more desirable balance of weights, and 
the like. 

However this may be, our present tests of intelligence 
are not merely instruments to measure how little stimulus 
is required to produce a perception of a relation, or how 
many relations will be perceived from a given constant 
stimulus, or how quickly. And we may best study them as 
they are before dismissing the valuations on which they 
are based, in favor of any simpler and more objective sys- 
tem. 

We shall then accept for the present the status of mea- 
surements of intellect as measures of different products 
produced by human beings or of different ways taken by 
them to produce the same product, each of these products 
and ways having value attached to it as an indication of 
intellect by a somewhat vague body of opinion whether 
popular or scientific. 

THE CONTENT OR DATA OF TESTS OF USTTELLECT 

Presumably a man can use intellect and display the 
amount of it which he possesses in operations with any 
sort of material object, any living plant or animal, includ- 
ing himself, any quality or relation that exists in reality or 
in imagination, any idea or emotion or act. Our tests 
might draw upon anything for their material. 

They have, in fact, greatly favored words, numbers, 
space-forms, and pictures, neglecting three-dimensional ob- 
jects and situations containing other human beings. How 
far this has been due to cc ivenience, and how far intellect 
is really best measured by As operations with words, num- 
bers, space-forms, and pictures, is a matter that obviously 
deserves investigation. Our choices of test material have 
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certainly been somewhat determined by convenience. They 
have also favored ideas, general notions, abstractions, sym- 
bols and relations, to the relative neglect of percepts and 
particnlars. This has been in the main deliberate, onr 
general scheme of valuation attaching on the whole more 
intellectual worth to operations with generals and facts in 
relations than to particulars and facts in isolation. 

The nature and extent of the specialization of intellect, 
according to the content or material operated on, has been 
and still is a matter of dispute; and the difference of 
opinion carries over into the practice of measurement. 
Some psychologists would be fairly well satisfied to mea- 
sure intellect by a series of mazes alone ; or by a series of 
sentence completions alone. Others, the great majority, 
attach much more confidence to a battery of tests including 
surely both words and numbers, probably also some space- 
forms and perhaps some more concrete pictorial material. 

THE FORM OF TESTS OF INTELLECT 

Whether we consider the external appearance of the 
tasks or the internal nature of the x>rocesses in the person 
doing them, there is a great variety in respect to form, 
that is, to the operations performed with the words, num- 
bers, pictures, and other content. Externally, there ap- 
pear questions to be answered, sentences or pictures to be 
completed, errors to be found and corrected, definitions to 
be given and to be chosen, items to be matched, directions 
to be followed, disarranged parts to be put together, dis- 
arranged events to be put in proper sequence, kej's or codes 
to be learned, true statements to be distinguished from 
false, items to be checked as fit by various criteria, items 
to be crossed out as unfit, and so on. 

Internally, the individual to at- 

tend to certain matters, to fix ^j^|PHhr3||[|pry, to recall 
what he knows about others, tq|^^B||^l|^^Mny things or 
ideas the one which best 6atisB^^H|npfe|^9Hiiremcnts, to 
define the relation between tTq^^^^HjgHHfcover an ele- 
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ment common to three or four given facts, to hold in mind 
many different facts and nse them to some specified pur- 
pose, and to inhibit customary habits in view of some rule. 
He also finds himself in some cases (such as many elements 
of information tests, vocabulary tests, and arithmetical 
computations) utilizing a wide range of knowledge and 
skills. 

Any system of units of measurement that is to be ade- 
quate must then apparently be flexible enough to apply to 
a wide variety of operations such as we may call attention, 
retention, recall, recognition, selective and relational think- 
ing, abstraction, generalization, organization, inductive and 
deductive reasoning, together with learning and knowledge 
in general. 


SCOBIHG THE PRODUCTS OF INTELiEOT 

In the great majority of instruments for measuring in- 
tellect the score or rating is determined in part by the de- 
gree of diflieulty of the tasks the individual can do success- 
fully. Thus * There are three main differences between a 
president and a king; what are they?’ (Stanford Binet 
XIV, 3) is harder than ‘Are you a little boy or a little 
girl?’ (Stanford Binet, III, 4). To complete 3 6 8 16 18 
36 ... . (Alpha 6, 20) is harder than to complete 10 15 
20 25 30 35 ... . (Alpha 6, 2). Psychologists and scien- 
tific and sensible people in general readily rank tasks as 
easy or hard for intellect and would accept the principle 
that,V^ Other things being equals the harder the tasks a per- 
son can master, the greater is his intelligence.*) The con- 
cept of hardness or difficulty in intellectual tasks, as now 
used, is somewhat vague and variable. Its outstanding 
characteristic is that among a large group of persons vary- 
ing in intelligence, the harder the task, the fewer will be 
the persons who can do it, and the more intelligent they 
will be. Sometimes, howel^er, tasks are called hard which 
really are only recondite, familiar to few ; and sometimes 
tasks are called hard which teally are only long. 
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We ahall presently define this concept of the inteOectual 
difficulty of a task, so as to make it more useful in science, 
but for the present we may leave it vague, the principle 
stated above being true^for any reasonable definitions of 
‘difficulty,* and ‘intelligence.* 

In many of the instruments for measuring intellect 
there are tasks which are of equal difficulty (or at least 
tasks so nearly equal that which of them is hardest is not 
certain). In the Binet series the tasks for any one year of 
age were supposed to be equally hard. In Alpha 7 only 
by statistical inquiry could one decide which of these is 
hardest, which next hardest and so on. 


6 love — hatred : : friend — lover mother need enemy 6 

7 wrist — ^bracelet : : neck — collar leg foot giraffe 7 

8 sailor — ^navy :: soldier — grm private army fight 8 

9 carpenter — Chouse : : shoemaker — hatmaker wax shoe leather 9 

10 shoestring — shoe : : button — coat catch hell hook — 10 ^ 

11 quinine — ^bitter : : sugar — cane sweet salt heets 11 

12 tiger — ^wild : : cat — dog mouse tame pig - 12 

13 legs — man ; : wheels — spokes carriage go tire. - 13 

14 north — south : : east — north west south east 14 

15 feather — :: rock — ages hill sink break 15 

16 grass— cattle : : bread — man butter water bones 16 

17 fin — fish : : wing^— /cather atr bird sail 17 

18 paper — wall : : carpet — tack grass sweep floor. 18 

19 food — ^mau : : fuel — engine h%im coal wood. - 19 

20 sled — ^runner : : buggy — horse carriage harness wheel. 20 

21 poison — death ; : food — eat bird life bad. 21 

22 Japanese — Japan :: Chinese — Evssia China Japanese pigtail 22 

23 angola — ^heaven : : men — earth woman boys JParadise 23 

24 Washington — Adams : : first — coKvtrost best second last 24 

25 prince — ^princess ; : king — palaoe queen president kingdom 25 


Bsks absolutely equal in 
|[y one person who gets 
^ get them all right, and 
i^ong will by no means 
^person’s score is partly 
is of equal difficulty that 


Now if a test includ^HIIHM 
difficulty for people in 
some right will by no m 
any one person who 
always get them all wro|§. M 
determined by the nun^^^i^iilji 
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he does. We must then consider as a possible principle 
* Other things being equals the greater number of tasks of 
equal difficulty that a person masters, the greater his 
intelligence,^ This principle would not be accepted so 
readily as the principle about greater difficulty, and per- 
haps would not be accepted at all unanimously. ‘Knowing 
more things than someone else, and being able to do more 
things than someone else’ is not so clearly and surely hav- 
ing more intelligence as ‘being able to do harder things 

than some one else can do.’ 

* 

The tw'o things have been somewhat confused in general 
discussions and in the construction of measuring instru- 
ments because, by and large, a person increases the num- 
ber of things he can do in large part by adding on harder 
ones, and also because the person who can do the harder 
can on the average learn those which the duller person can 
* learn more quickly than he, and so learns more of them. 
Consequently what we may call llie lei' el or height or alti- 
tude of intellect and what we may rail its extent or range 
or area at the same level are correlated and €‘ither one is 
an indicator of the other. It will be best, however, to keep 
them separate in our thinking. 

In many of the instruments for measuring intellect a 
person’s score is determined partly by the speed with which 
he can do the tasks. Even in batteries of tests where all 
candidates attempt all the tasks, speed may count, since the 
persons who do the easier tasks more quickly may have time 
to j'eview some of the tasks and perfect their work. If 
speed deserves any weight in determining the measures of 
intellect it is by virtue of the principle that, ‘Other things 
being equal, the more quickly a person produces the correct 
response, the greater is his intelligence/ Giving much 
weight to speed arouses decided objections in the laity and 
among some psychologists, and the principle just stated 
certainly would not be accepted as axiomatic. By and 
large, however, if A can do harder things than B can, A will 
do those things which B can d ) more quickly than B can 
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A certain moderate weight attached to speed ■vnll not th^isi 
much decrease a test’s significance; and, i)er hour of 
spent on testing and scoring, an even greater significobis^ 
may perhaps be obtained by giving a liberal weight for 
speed than by giving none. 

For the practical purposes of estimating intellect, a 
battery of tests in which level, extent, and speed combine in 
unknown amounts to produce the score may be very useful. 
For rigorous measurements, however, it seems desirable to 
treat these three factors separately, and to know the exact 
amount of weight given to each when we combine them. 

We shall try to make the concepts of intellectual prod- 
uct, difficulty of producing an intellectual product, range 
of products produced, and speed of producing a product, 
more definite and precise, but without so altering them as 
to lose the elements which have given them practical value 
in the best current practice in measurement, or to weaken 
in any way their usefulness in measuring intellects as we 
actually find them by the tests which we have so far de- 
veloped. 

We shall start with certain first approximations. For 
a first approximation, let intellect be defined as that quality 
of mind (or brain or behavior if one prefers) in respect to 
which Aristotle, Plato, Thucydides, and the like, differed 
most from Athenian idiots of their day, or in respect to 
which the lawyers, physicians, scientists, scholars, and edi- 
tors of reputed greatest ability at constant age, say a dozen 
of each, differ most from idiots of that ago in our asylums. 

Let an intellectual product, i.e., a product or response 
requiring, or depending on, intellect for its production, be 
defined as a product or response which, given the same ex- 
ternal situation, the intellects in the half toward Aristotle 
are more likely to make than the intellects in the half to- 
ward the idiot. For example, if, when all Athenians of 
age forty were confronted by the question ‘Is a straight 
line the shortest distance between two points t’ the growth 
of the white blood corpuscles was equal for the Aristotelian 
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and the idiotic halves, whereas the answer Yes was more 
prevalent in the Aristotelian half, we should rate the latter 
as a product depending on intellect, and the former as a 
product not depending on intellect. 

Let the intellectual difficulty of producing a given intel- 
lectual product in response to a given external situation be 
defined as follows : Enough time being allowed for produc- 
tion so that an increase in time would not increase the num- 
ber producing it, the difficulty for Athenians of forty is 
approximately greater the smaller the number of them who 
produce it, provided that the ranking of those who do pro- 
duce it differs from the ranking of those who do not by 
greater nearness to the Aristotelian end. We could be 
much more rigid here by supposing a population to vary 
from the idiots to the Aristotles in amount of intellect only, 
being identical in all else. Then, if all conceivable pro- 
ductions of intellectual products in response to given ex- 
ternal situations were ranked for difficulty, the order would 
be very closely that of rarity and of the nearness to Aris- 
totle of those who achieved it. We could omit the ‘ap- 
proximately,’ and the ‘jjrovided that.’ Our definition has 
deliberately been left loose, since we do not know exactly 
what it is in which Aristotle differs most from the idiot, 
much less can we know in the case of any group of actual 
individuals that they are identical in all else than it. 

The range of products produced at any one level, i.e. 
of products which are equally hard to produce, is defined 
simply by their number. What we may call the relative 
range at any level may be defined as the percent or fraction 
of the products at that level which can be produced by the 
intellect in question. The speed of producing any given 
product is defined, of course, by the time required. 

It will be convenient to use the word task to mean the 
production of a given product in response to a given exter- 
nal situation, and to speak of the difficulty of tasks, the 
number of tasks of a given difficulty that can be done, and 
the speed of doing a given task. 
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We now have intellect defined by a ranking of men whose 
differences therein are roughly appreciated as we appre- 
ciate the differences of the world’s varied objects in volume 
(only much more roughly.) We have intellectual tasks and 
products defined in a catiiolic way that would, for instance, 
probably include every task in all the stock instruments in 
use by psychologists to-day. We have difficulty defined 
objectively so that a series of tasks could be approximately 
ranked as to their respective amounts of difficulty for any 
specified group. 

If we list all tasks, find the difficulty of each, apply an 
intellect to them,, observe which it can do, and how long it 
requires to do each, we have measured how hard tasks it 
can do, how many it can do at each level, and how quickly 
it can do them. If we use in place of a complete list of 
tasks a fair sampling from them, we have attained the same 
end, subject to the error of our sampling. 

The new problems of theory and technique in the mea- 
surement of intellect, that is, the problems not soluble by 
the general methods of measurement in any science, con- 
cern the measurement of difficulty of task. Extent and 
speed are measurable in two of the most perfect units there 
are — number and time. In the case of difficulty, however, 
we have so far provided only for an inventory of intellec- 
tual tasks and their arrangement in an order of difficulty. 

Their differences in amount of difficulty and the dif- 
ferences between the amount of difficulty of any one of 
them and some zero point of difficulty (some task which is 
just below a task of infinitesimal difficulty), are not deter- 
mined. To find ways of determining these will be our 
R "'n wotk. 

lef^'ire attempting it, however, we may best consider 
a further facts about difficulty, extent and speed in 
the production of intellectual products, and certain conse- 
quences of our analysis of a measurement of intellect into 
this three-fold determination. 
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FURTHBB FACTS OOHCERNINQ DIFFICUtiTY 

We have defined intellectual difficulty in relation to a 
defined group of individuals. How far the rank order for 
difficulty obtained in the case of one human group will hold 
for others, or for a group of dogs or of chickens, is a mat- 
ter better ascertained by experiment than prejudged. Diffi- 
culty in our treatment is always difficulty for some specified 
group of intellects, such as our Athenians aged forty. We 
can, if we wish, specify the group as all human beings of 
all ages, or all animals, and so get measurements of some- 
thing which we might call difficulty in general. The value 
of such a measure will, however, depend largely on the close- 
ness of correspondence between the rank orders for the 
same series of tasks at different ages, in different civiliza- 
tions, and so forth. If these are very low, the measure- 
ment of such difficulty in general may be of very little use. 

Many cases of grouping, as by age, by amounts of gen- 
eral education, by amounts of special education, or by city 
and country environment, are of great importance. Two 
may be considered briefly now as samples, namely, group- 
ing of those of equal chronological age by amounts of in- 
tellect, and grouping of those of equal intellect by chrono- 
logical age. If certain tasks are of difficulty k, k-\- a, 
k-\- a-\-h, k-{- a-\-h c, etc., for 12-year-olds of low or 
small intellects, say the bottom tenth of twelve-year-olds, 
how far will they retain the same relations in respect of 
difficulty in the case of the top tenth ? If certain tasks are 
of difficulty fci, ki-\- a, kt-\- a + h, etc., for the eight-year- 
olds of a certain degree of intellect, how far will they retain 
the same difficulty relations for sixteen-year-olds of the 
same degree of intellect ? 

We have eliminated speed entirely from influence upon 
the measurement of difficulty, by our condition that such a 
time allowance be given for the task that no further increase 
in time would alter the production. In practice, this would 
only be approximated. Obviously we must not make the 
time so long that during it the intellect in question changes 
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appreciably by growth or training. We should not leave 
individuals to strive for ten hours to complete : ‘ The body 

gives light the is the 

sun,* because once in ten thousand times, some child who 
failed during nine hours succeeded in the tenth. This would 
be a valuable experiment, but we have far more valuable 
ways of using ten hours of his time. 

What we are really concerned about is to avoid rating 
one task as harder than another merely because it is longer, 
so that the poorer intellects do it less quickly than the 
others, and so, within a too short time limit, show a spuri- 
ously greater percentage of failures. 

We have made the requirement that the intellectual 
ranking of those who do produce the response shall be 
higher than that of those who fail. Usually this require- 
ment is uimocessary. It can, that is, usually be assumed 
that the good or correct response wall be obtained by the 
better intellects more often than by the poorer. It is inserted 
to provide against cases where the better intellects are sub- 
ject to some constant error so that they give fewer correct 
responses than the dull do, or where other factors than in- 
tellect distort the percent of rights from what it w^ould be if 
everything but intellect were equalized. For example, it is 
conceivable that, if (a) and (h) below were given to a ran- 
dom sampling of intellects, 

Underline the right answers: 

(а) 4"* equals ^ 3 5 41 

(б) 4^ equals 2 3i 8 412 

ratings for difficnliy by the percents correct would be very 
much in error. The percent for (a) w^ould probably be 
lower than for (b) because, lacking knowledge of exponents, 
the more intelligent one was, the more likely one wmuld be 
to report 3 for (a), (valid if 4r^ means 4 — 1), and to report 
2 (valid if 4i means 4 halves or 4 X i) for (b). 

We have treated the task as being to produce a certain 
product. It is scored, consequently, as done or not done. 
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success or failure, right or wrong. Now when any task for 
intellect is set there are often many different responses 
varying in ‘goodness’ or correctness. In such cases, our 
method requires that in determining the difficulty of the 
task, a dividing line be set somewhere.® Our method will 
not, however, prevent us from later using different credit 
values in a scoring plan for such a task and taking full ad- 
vantage of whatever added value these more detailed credit 
values may have in estimating an individual’s intellect. 

It may be noted further that a task may consist of vari- 
ous combinations and complications of other tasks. Thus 
the task may be to get the right answer to 8 + 3, or to get 
the right answer to 11 + 7, or to get the right answers to 
both 8 + 3 and 11 + 7, or to get the right answers to 8 + 3 
and 11 + 7 and also 18 + 4, or to get the right answer to : 

Find the sum 

9 

4 

7 

3 

8 

which ordinarily involves the above, plus knowledge of 
22 + 9, of certain words and procedures, and control over 
certain habits, such as holding numbers in mind, and adding 
a seen to a thought-of number. 

We arc now in a x>osition to state one theorem of the 
measurement of intellect. Let difficulty be defined as above, 
then: 

Theorem I: Other things being equal, if intellect A can 
do correctly all the tasks that intellect B can do save one 
and in place of that one c£.n do one that is harder than it 
intellect A has the higher level. 

One is tempted to go further and assume that, other 
things being equal, if A and B can do correctly the same 
number of tasks, A has the higher level, if the average diffi- 

® What seems to be one task to the person tested may be used as two or 
more tasks by scoring it drst with the dividing line at one place, and second 
with the dividing line at another. 
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culty of the tasks he can do is greater than the average diffi- 
culty of the tasks B can do. This cannot, as yet, be widely 
assumed first because we do not know that we have any 
right to average measures of difficulty,® and secondly, be- 
cause, even if we could, it is not safe to assume that as much 
intellect is required to do 10 tests each of difficulty 20 as to 
do one task of difficulty 200. 

On the other hand one is tempted to suggest the measure- 
ment of an intellect by the hardest things it can do, assuming 
that since it can do these, it could do all easier, as we assume 
that one who can jump over a bar 6 feet high could surely 
jump over bars at 5 ft. 10 in., 5 ft. 8 in., and so on. The 
possible variety and specialization of intellectual tasks 
makes this uncertain. 

WIDTH OR EXTENT OR RANGE 

Our definition of greater difficulty enables us also to 
define equal difficulty and so to make a fairly rigorous defini- 
tion of width or extent or range by making it separately at 
each level of difficulty. For any specific group G and any 
specific time t those tasks are equally difficult which are done 
correctly by equal percentages of intellects. 

Consider then all the tasks which are of a certain diffi- 
culty D. Some intellects viull fail with all of them. Among 
the intellects which succeed with some of them we may make 
comparisons according to the number succeeded with. Such 
a statement as tasks, of equal difficulty D, being given, 
with t time allowed per task, A did O.IN while B did 0.2 jV 
and C did 0.3A,' is clear and useful. We can say that B 
did twice as many as A, that C exceeded B in the number 
done as much as B exceeded A, and that the average for A 
and C was the same quantity as the score for B. Where the 
problem concerns the extent of an ability, as in the number 
of certain facts that are known in history or science or the 

B We have provided for determinations of wMch one of two or more tasks is 
the more difficult, but not, as yet, for determinations of how much more diffi^ 
cult it is. 
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number of certain procedures that are mastered in arith- 
metic or carpentry, it is often, perhaps usually, desirable to 
free the measurement from differences in difficulty by mak- 
ing the tasks equal in difficulty and measuring extent at 
that level,^® 

In the measurement of intellect, measurements of extent 
at each level are obviously instructive for many purposes. 
The inventory of what intellect A can do is improved by 
being classified into Tasks 1, 2, and 4 at level Di, 16, 19, 27, 
and 28 at level Dg, 37, 43, 48, 49, and 56 at level Da, and so on. 

We can set down as Theorem II: Other things being 
equal, if intellect A can do correctly all the tasks that intel- 
lect B can do, and can also do one more task at the level of 
any of the othei's, intellect A has a greater range than intel- 
lect B has. We could also safely say that A is a better or 
more useful intellect ; whether we can rightly say that it is 
a greaier than B is more doubtful. The latter seems to im- 
ply that superiority in extent can be made commensurate 
with, weighed in the balance against, superiority in level. 
For the present let us leave the question open. 

SPEED 

There is of course no essential difficulty in measuring 
the time required for intellect A to produce a certain prod- 
uct. Number and time figure in mental me}i.surements as 
they do in physical measurements. The units of number 
and time are indeed so much more convenient and intelli- 
gible than units of difficulty that there is a strong natural 
tendency in those who devise instruments for measuring 
intellect to let their measurement depend largely upon the 

It should be noted that a nuinber of tasks of equal dlfficultj may be given 
in a test instrument, not with any intention of measuring extent of intellect at 
that level for its own sake, but simply in order to obtain a more accurate 
measure of the level itself. For example suppose that in instrument x we have 
tasks at ten levels one at each. Suppose that in instrument y 

wo have ten at each level. Then by whatever convention we determine how hard 
a task a peiuon can do, we shall determine it much more exactly by instrument y 
than by instrument x. 
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ntunber of correct responses and the speed of producing 
them. 

In the instruments that are actually used, it is customary 
to have the time a mixture of (1) the time spent in doing 
some tasks correctly, (2) the time spent in doing other tasks 
incorrectly and (3) the time spent in inspecting other tasks 
and deciding not to attempt them. This confusion may 
be permissible, or even advantageous, in the practical work 
of obtaining a rough measure of intellect at a small expense 
of time and labor and skill, but for theory at present and for 
possible improvement of practice in the future we need to 
separate the speed of successes from the speed of failures. 

To the number of tasks eorrectlj'' done at each level we 
may add a record of the time for each or of the average 
time for all at that level. 

Since to save time in intellectual production is a ‘good,’ 
we may frame Theorem III as follows: Other things "being 
equal, if intellect A can do at each level the same number of 
tasks as intellect B, hut in a less time, intellect A is better. 
To avoid any appearance of assuming that speed is com- 
mensurate with level or with extent, we may replace ‘better’ 
by ‘quicker.’ 

THE RELATIVE IMPORTANCE OF ALTITUDIiJ, EXTENT, AND 
QUICKNESS OF INTELLECT 

Each of these three factors is essential. If it required 
an infinite time per task, an intellect would produce no prod- 
uct at any level no matter how high its potentialities as to 
altitude and extent might be. If it had zero extent at all 
levels, it would not matter how high its potentialities as to 
altitude or how quickly it could do nothing. In the ordinary 
sense of the word, however, altitude or level is by far the 
most important. The chief evidence for this is that it alone 
is indispensable, irreplaceable by anything save itself. If 
the best available intellect can do only things of level D,», 
we cannot get things of level Djo done at all. If the best 
available intellect can do only 72 things at level D* and we 
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need to get 144 things at that level done, we need only to 
get other intellects at work, say one that can do 45 of the 
balance and another that can do the remaining 27. If the 
best available intellect can do only 10 tasks per minute at 
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Fig. 3. The measure of a superior intellect. 

level Ds and we need to get 20 done per minute, we can hire 
five common people who can do two a minute to help. In- 
deed, we shall be wise to hire ten common people to do two 
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a minute each, and leave the best available intellect to put 
its time on tasks far above level D*. 

Common sense recognizes the greater importance of alti- 
titude. It rates a Pasteur far above the most widely com- 
petent general practitioner. It does not ask how quickly 
Milton could give opposites, or turn out doggerel rhymes. 
Probably Pasteur was very much above the average in 
extent of intellect; probably Milton could have written as 
good poetry as A can write very much faster than A can. 
But common sense considers extent and quickness as unim- 
portant in comparison with reaching a level far above the 
average. 

From the economic and philanthropic points of view, 
altitude is enormously more important. If an intellect 
could be hired from Mars of so high level that it could learn 
how to prevent war as easily as Jenner learned how to pre- 
vent smallpox, a million dollars a day would be a cheap 
wage for the earth to pay him. 

Our analysis of the measurement of intelligence may be 
represented by space and number as follows : 

Let one sixteenth of a square inch represent one intellec- 
tual task. Let those equal in difficulty be placed in the same 
row across the page; let the order of the rows from the 
bottom to the top of the page be the increasing order of 
difficulty ; let the square be shaded if the individual in ques- 
tion cannot do it ; if he can do it, let it bear a number repre- 
senting the time he requires to do it. For illustration, we 
have assumed that there are 320 tasks and that they are of 
20 levels of difficulty, 16 at each level. 

Figures 3 and 4 then represent the measurements or in- 
ventories of two specimens of intellect. Such measurements 
or inventories may be abbreviated by using a random sam- 
pling of tasks at each level, or by using only every other 
level or every third or every fourth level, or in other ways. 
Only one thing is needed to make such measurements sub- 
missible to the arithmetic and calculus of science in general. 
That is the expression of the altitude of each level (now 
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merely a rank) as an amount of difference from the altitude 
of the others and from some group of tasks which require 
intellect, but so little of it that they border on a true zero 
of difficulty which may be set as their lower limit. This is 
the fundamental problem of mental measurements. 



Fio. 4. The measure of au inferior intellect. 






















































CHAPTEB n 

The Measurement op Difficulty 

THE PRESENT STATUS 

If the members of a group are tested with each of a num- 
ber of intellectual tasks, these can be put into an approxi- 
mate rank order for intellectual difficulty for that group by 
tlie liercentage of successes. For example, tasks A to G 
below are arranged by steps in an approximate order of 
intellectual difficulty for the group, “persons at the time 
of graduation from Grade 8 in City A in 1924.** 

The same sort of validity that attaches to the statement 
that G exceeds A in intellectual difficulty for that group, 
attaches also to the statement that G so exceeds P, that F so 
exceeds K, and so on. 

Write words on the dotted lines so as to make the whole 
sentence true and sensible. AVrite one word on each inch 
of dots. 

A. A strong man lift a heavy box. 

B. The rose is a favorite because of 

— -fragrance and 

(.^ A body of entirely surrounded by 

.is called an 

D. It may effort and a long. 

but the result is sure. 

E. You may safely conclude that you in 

yourself the means of at the truth. 


F. He believed i n h ard things 

.because .hard. 

G. Judicial decisions are of or less author- 


ity as precedents to circumstances. 

The same sort of evidence and argument which decides 
that G has more intellectual difficulty than A, was used to 
place B, C, D, E, and F in order between A and G. The 
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evidence is the number and natnr^, in respect of iintellect, 
of those who succeed with each tas^t. The harder Ime task, 
the fewer the persons who succeed ra it, and the more intel- 
lect they have. The argument implicitly involved is (1) that 
whether a person succeeds or fails in such tasks is deter- 
mined largely by the amount of intellect which he possesses, 
and not greatly by anything other than intellect, and (2) 
that in the hardest tasks which a person masters, he uses 
in general nearly all the intellect which hC'-has. 

The argument is sound enough to justify such a rank 
order as the A, B, C . . . G order shown above, or the 
order of a series made of Stanford Binet tests for Mental 
Age 10, Mental Age 12, Mental Age 14, Mental Age Adult, 
and Mental Age Superior Adult, but we shall find trouble 
if we try to make a very close ordering, or to use the per- 
centages of successes for other than approximate rankings. 

The exact determination of a rank order of test elements 
for intellectual difficulty requires that the individuals in the 
group be tested with each of the tasks under similar condi- 
tions, including interest and effort, which is a matter of 
general scientific care that needs no further discussion here. 
It requires also that each of the tasks in the series shall be 
‘intellectual;* and this requirement will eventually need 
very elaborate discussion. We shall, indeed, find that it is 
desirable to define an intellectual task as one in which the 
person tested uses all the intellect he then has ; and in which 
he differs from other persons in nothing save the amount 
of intellect used. 

If, however, we applied any such rigorous definition now, 
we should be unable to deal with any elements of any tests 
ever used in measuring intellect, since not a single one of 
them is a task which depends on intellect in its entirety, 
and differentiates individuals with no disturbance by any- 
thing other than intellect. A test element which did so 
would correlate 1.00 with a perfect criterion. In order to 
maintain continuity with previous work, we shall first treat 
each test element as if correct response to it was caused by 
intellect intact and uncontaminated by aught else. 
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Two methods have been used to measure the difficulty 
of intellectual tasks. The first has used the iudfa^enia of 
teach ers, psychologists, and other judges of presumable 
competence, relying on -some assumptions such as that if 
K percent of competent judges rate A as harder than B, 
and B as harder than C, the differences in difficulty, A-B 
and B-C, are equal. We shall report our investigations of 
this method later ; for the present, it may ^^e disregarded. 

The second has used the percentages of some defined 
group of individuals who succeed with each task, relying 
on some assumptions about the form of distribution (in 
that group) of the ability involved in doing such tasks. 

For example, suppose that we knew that ten thousand 
individuals were, in respect to levels of intellect, distrib- 
uted as follows: 


At 

level 

X 


10 

i c 

i i 

X-h 

k 

100 


< < 

x-f 

2k 

1000 

n 

a 

X + 

3k 

1190 



X-1- 

4k 

1000 

it 

< i 

X-f 

5k 

1000 


n 

X + 

6k 

2000 

i i 

4 4 

X + 

7k 

2000 

IC 

4 4 

X-i- 

8k 

1000 


4 4 

X + 

9k 

500 

iC 

4 4 

X 10k or higher 

200 


and suppose that the ten thousand were tested "with various 
intellectual tasks with the following result: 


10000 had 

Task 

No. 

6 right 

9990 

44 

44 

44 

18 right 

9990 

4 4 

4 4 

4 4 

19 right 

8890 

4 4 

44 

44 

15 right 

8890 

4 4 

4 4 

4 4 

20 right 

6700 

44 

4 4 

44 

22 right 

3700 

44 

44 

44 

28 right 

1700 

44 

44 

4 4 

29 right 

700 

44 

44 

4 4 

30 right 

200 

44 

44 

44 

31 right 

200 

44 

44 

44 

33 light 




5 
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Using these results at their face value, we should con- 
clude that within this group No. 6 can be done by intellects 
of level X, and perhaps of lower levels ; Nos. 18 and 19 re- 
quire level X + k; Nos. 15 and 20 require level x + 2k; No. 
22 requires level x 5k; No. 28 requires level x + 7k; No. 
29 requires level x + 8k ; No. 30 requires level x + 9k; Nos. 
31 and 33 require level x + 10k or higher. Nos. 31 and 33 
are thus Ik harder than 30, which in turn is Ik harder than 
29, which in turn is Ik harder than 28, which in turn is 2k 
harder than 22, which is 3k harder than 15 or 20, and so on. 

THE MEASUBEMENT OF DIFFEKENCES IN DlFFlCUIiTY BY WAY 
OF KNOWLEDGE OF THE FOBM OF DISTRIBUTION OF 
THE VARIATIONS OF AN INDIVIDUAL IN 
LEVEL OF intellect' 

An individual does not display the same level of intellect 
at all times and seasons. He varies around his average 
status. If we know the real form of distribution of his 
variations in level, we can use it to compare the differences 
of tasks in real difficulty, just as we use knowledge of the 
form of distribution of a group. 

1 We began our search for means of measuring differences in. difficulty by 
inquiring whether the real form of distribution of the real abilities of the indi** 
viduals represented in a simgle array in a correlation tahle, might not be de- 
termined with greater certainty than the form of distribution of the group as a 
whole. This is indeed often the case ; and the use of a group sorted into arrays 
has much to recommend it. The consideration of the factors which do influence 
the form of distribution of the real ability of the individuals in an array, led 
us to a bi'^ader view of the means of scaling difficulty of task and level of 
intellect. 

The form of distribution of the real alulities in an array is determined by 
three causes: (1) The form of distribution of an individual’s variations around 
his own average; (2) the relation of an individual’s variability to his amount 
of ability, and (3) the form of distribution of the entire group from which the 
array is sorted out by its correlation. 

It will be shown that if we can determine the facts for any one of these, 
we can transmute certain differences in rank into differences in amount. The 
transmutations by (1) and (2) are almost, if not quite, independent of those 
^7 (3) respect of facts and assumptions, and so provide a check of great 
value. The use of an array instead of a total group utilizes all three methods 
together in a way that has many advantages. We shall not, however, make 
use of this method in the main body of our work. 
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Suppose, for example, that an adult is measured with a 
hundred examinations, each consisting of 100 tasks, each 
of ten levels, Ij, I 2 , la, I 4 , etc., of increasing difficulty. Sup- 
pose that the highest level at which he attains 50 percent 
correct® is level I 3 in 27 cases, level I 4 in 50 cases, and 
level la in 23 cases. Then, if we know the form of distribu- 
tion of his variability, we can compare I5-I4 with U-la in 
terms of amount. We could thus put into comparison the 
differences between any two levels between his upper and 
lower limits. 

This method will be very useful as a check upon estimates 
of difficulty via the form of distribution of a perfectly mea- 
sured group, because it is so independent thereof. The 
forces which make one individual vary from one time to 
another are probably almost, if not quite, different from 
the forces which make one individual vary from other in- 
dividuals when all are perfectly measured. 

We have therefore studied the variability of an individ- 
ual in repeated tests of intelligence and other mental abili- 
ties at some length, trying to discover the form which its 
distribution would have if the ability in question were mea- 
sured by a scale of truly equal units, instead of the arbi- 
trary scales which we have. The results, which appear in 
Appendix T, are substantially unanimous in fitting the hyi>o- 
thesis that, omitting such extreme conditions (for example, 
being asleep or being seriously ill) as prevent an individual 
from being tested at all, depressing conditions are neither 
more frequent nor greater in their effects than elevating 
conditions, the real variability being symmetrical. 

For example, 60 pupils in Grades 4, 5, and 6 were tested 
with Stanford Binet, National A, National B, Otis ad- 
vanced,® Myers Mental Measure, Haggerty Delta 2, Illinois, 
and certain parts of Dearborn. Each score was first turned 
into a deviation from the median for the group in that test, 

2 Or 60%, or 90%, or whatever percent we are using as a measure in our 
experiments. 

8 Some had the Otis Primary instead* For these, estimated scores in the 
Otis Advanced were computed. 
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in terms of the variability of the group in the test in ques- 
tion. Then it was expressed as a deviation from the aver- 
age of the eight such deviation-scores for the individual in 
question. These last deviations represent the variability 
of an individual around his average ability in intelligence 
tests. Their distribution is as follows : 


Deviations 
"165 to "194 
"135 to "164 
"105 to "134 
" 75 to "104 
" 45 to " 74 
" 15 to " 44 
" 14 to + 14 
+ 15 to + 44 
+ 45 to + 74 
+ 75 to +104 
+ 105 to +134 
4 135 to + 164 
+ 165 to +194 


Frequencies 

0 

2 

1 

9 

40 
124 
137 

86 

41 
10 

3 

2 

1 


65 pupils in grades 8 to 12 were tested with Alpha Form 
5, Alpha Form 8, Terman Group Test Form A, Terman 
Group Test Form B, and half of Part I of the Thorndike 
Examination for High-School Graduates. These five scores 
for each pupil were treated just as the eight scores de- 
scribed in the previous paragraph, except that the final 
deviations are deviations from the individual’s median in- 
stead of from his average. 

The resulting distribution was as follows (including the 
65 zero deviations of the medians themselves) : 



Deviations 

Frequencies 

"110 

to 

" 

129 

1 

" 90 

to 

" 

109 

2 

o 

! 

to 

" 

89 

10 

O 

1 

to 


69 

10 

- 30 

to 


49 

34 

" 10 

to 

" 

29 

47 

- 9 

to 

+ 

9 

11" 

+ 10 

to 

+ 

29 

05 

+ 30 

to 

+ 

49 

27 

+ 50 

to 

+ 

69 

15 

+ 70 

to 

+ 

89 

5 

+ 90 

to 

+ 109 

2 

+ 110 to 

+ 129 

0 
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We can then use the method to compare amounts of diffi- 
culty up and down from an individuals average level. Our 
results also seem to fit the hypothesis that the real variabil- 
ity of an individual, under the conditions stated above, fits a 
probability surface limited at about and — 3 S.D. better 
than it fits any other one surface. They are not, however, 
such as justify a rigorous quantitative treatment of fit. 

THE RELATION OF THE VARIABILITY OF AN INDIVIDUAL TO HIS 

AMOUNT OF ABILITY 

The facts of the previous section enable us to compare 
and equate differences in difficulty for an individual of aver- 
age level Ik, using of course many such individuals to gain 
exactitude, or general reference, or both. These differences 
will be restricted in range to levels not very far below or 
above Ik. If we wish to compare and equate them with dif- 
ferences outside that range, we must use individuals of 
average level higher than It and individuals of average 
level lower than h. To use them we must know how much 
less or greater their real variability is than that of our 
starting group with average at h- 

We have, therefore, made very extensive investigations 
of the relation of an individual’s amount of variability to 
his amount of ability. These are reported in Appendix II. 
Only their general nature and results will be stated in this 
section. Here as elsewhere we distinguish sharply between 
(a) the apparent, or face- value, relation observed between 
the variability of an individual’s separate scores and his 
average score, and (b) the real relation that would be ob- 
served if these scores were transmuted into measures such 
that 1, 2, 3, 4, 5, 6, 7, etc., represented a real arithmetical 
progression of amounts of the ability. For example, we 
find that twenty individuals each of whom took (after two 
preliminary trials, to eliminate the practice effect) from 
eleven to thirteen forms of Part I of the Thorndike Intelli- 
gence Examination for High School Graduates, showed the 
results of Table 1. If the scores are taken at their face 
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value, it appears that the variability of an individual whose 
median score is about 105 (from 100 to 113) is very nearly 
the same as the variability of an individual whose median 
score is about 128 (125 to 132). If, however, the units of 
the scoring scale from 90 to 120 really represent smaller in- 
crements of ability than the units from 120 to 145, the real 
variability of an individual of ability 105 is less than the 
real variability of an individual of ability 128, and con- 
versely, if the units of the scoring scale from 90 to 120 
really represent larger increments of ability than the units 
from 120 to 145. 

We thus record the face-value-score results for many 
different sorts of tests of intelligence,* noting in each case 
any facts about the construction of the tests which concern 
the probability that its units progressively swell or shrink 
in ‘real* value over any considerable fraction of the range 
we are concerned with. We note especially the results in 
those cases where there is no reason to expect swelling 
more than shrinking. The average relations between varia- 
bility and ability found in these cases may be taken to rep- 
resent approximately the real relation, until some one pro- 
duces evidence that, in all or nearly all tests for the ability 
in question, there are forces leading psychologists, quite 
without intention, to devise scoring plans which make for 
progressive swelling or shrinking of units. 

The general drift of the facts is shown in Table 2 which 
gives the variability (in face-value-score units) of an indi- 
vidual from day to day in intellect as a percent of the varia- 
bility of a person whose amount of intellect is that repre- 
sented by an Army Alpha first-trial score of about 100. 

« We have secured extensive data concerning Army Alpha, Examination A, 
Army Beta, Stanford Mental Age, the National Intelligence Test, the Otis 
Advanced Test, the Haggerty Delta 2, the Myers Mental Measure, the Kelley- 
Trabue, the Stanford Bluet, the Terman Group Test, the I.E.B. Test of Selec- 
tive and Belational Thinking, the I.E.B. Test of Generalization and Organiza- 
tion, the Thorndike Non-Verbal Test, the Thorndike Examination for High 
School Graduates, series of 1919 to 1930, and the Toops Clerical Test. See 
Appendix II. 



TABLE 1 

Vabiations of the Scores of Thirteen (or Fewer) 30 -Minote Trials with Part I of the Thorndike Intelligence Exauina- 
TioN FOB High School Graduates from the Median Score for the Individual in All Thirteen 
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It appears from Table 2, and still more clearly from the 
consideration of the detailed facts in Appendix II which 
Table 2 barely summarizes, that if we had scales for 
intellect whose units were really equal, the variability of an 
individual from day to day would be the same, rej^ardless of 
whether the average amount of intellect possessed by him 
was that of a ‘low grade ten-year-old’ or of a ‘superior 
adult, ’ that of an Army Alpha score of 25 or that of a score 
of 175. 

This result is so important, if true, that we have sought 
for facts and probabilities in real or apparent opposition 
to it. 

First, there are the obvious opposing facts of range of 
variability in intellectual or similar production. Keats may 
have written “Ow Reading Chapman’s * Homer’ ” in one 
hour, and have written nothing in some other hour when he 
tried as hard, whereas an average twelve-year-old varies at 
the most from nothing up to a composition scoring 50 on 
the Hillegas scale. A gifted stock-exchange trader who in 
transactions of 10,000 shares a day, averages $100 profit, 
may vary from a profit of $25 to one of $2,500, whereas a less 
gifted trader who averages $10 a day on 100 shares in the 
same market, it is said, varies over a much narrower range. 

Such apparently opposing facts as these are, however, 
not so simple as they seem. If we had a full record of all of 
Keats’ hours of equal effort, the production called zero 
might turn out to be far above zero. The ideas he had then 
might rank in poetic value far above those of the best hours 
of the average man. The less gifted trader may vary over 
just as wide a range. For example, a still less gifted trader 
losing $100 on the average, may lose in two days the $25 
and the $2500 that the gifted trader gains. Furthermore, 
we have to consider the alleged common observation that 
as one increases his expertness in acting, music, dancing, or 
athletic feats, he seems to reduce his variability. Thus a 
sprinter ■who can on the average run 95 yards in 10 seconds 
almost never runs less than 90 yards or more than 98 yards 
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in that time, whereas the heavy-footed youth whose average 
is 50 yards in 10 seconds may reach 60 and fall below 40. On 
the whole, what facts there are concerning the relation of 
variability to amount of ability in mental traits in the world 
at large, seem to favor a slope down about as much as a 
slope up. 

Certain hypotheses concerning the constitution of in- 
creases in amount in the case of mental traits are in oppo- 
sition to our result. One such hypothesis is that there are 
certain factors producing intellect all of which act positively 
and by addition. For simplicity's sake, we wiU assume that 
each factor contributes as much to intellect as any other. 
Then the average amount of intellect that an individual dis- 
plays depends on the number of these factors that he pos- 
sesses, and his variability from time to time depends on how 
many of them are then active. Assume that each of them has 
the same probability of acting as any other and that the num- 
ber of them that will be active at any one time is a result of 
the probabilities of their several combinations. In such a 
ease the variability increases as the square root of the aver- 
age amount of the ability. For example, if there are five, 
ten, fifteen, and twenty factors in individuals A, B, C, and 
D, respectively, the status of A, B, C, and 1) at a thousand 
points taken at random in their lives, will be as shown in 
Table 3, and the variability of D will be twice that of A. If 
intellect is caused in any such way as this, the number of 
factors is probably very large so that we may better change 
our illustration from 5 and 20, to say 1,000,000 and 1,500,- 
0 00. The variabilities are then (mean square deviations) 
V 250,000 and V375,000.* The ability which is 500,000 
greater, or 1^ times as great, has a variability which is only 
1.2 times as great.® 


® These are for points of time. The variabilities in testa covering thirty 
minutes or so are the variabilities of averages of many such points^ and may 

be represented as ^ — ■ ■■ and ^ ^^^^£2^where n is relatively large, say 

V n V n 0 7^ 

IfOOO; this, however, does not change the relation of variability to amount of 
ability. 



THE MEASTJBEMEXT OP DIPPICrOX.TY 


49 


Siidi hypotheses as this can be nearly reconciled with 
our results if the difference between the intellect of the level 
of Alpha 25 and Alpha 175 is due to an increase in the 
number of factors which. is large absolutely, but small in 
comparison with the number involved in Alpha 25. Thus if 
the difference is 500,000, but an ability of Alpha 25 involves 
5,000,000 then the variabilities around levels of 25 and 125 
will be as Vi, 250, 000 and V 1^350,000, or as 1118 and 1162, 
the second being only 4 percent greater. The reasonable- 
ness of this depends upon the location of the absolute zero 
of intellect. If that is ten times as far below Alpha 25 as 
Alpha 25 is below Alpha 175, it is perfectly reasonable. 

Another way out of the difficulty is to deny the validity 
of the theory that intellect is constituted by the addition of 
positive factors only. If the factors in the above illustra- 
tions were inhibitive against some maximum amount of 


TABLE 3 

The Variability op Four iNMviDtrALB in Intellect According to a Certain 
Additive Combination op Factors All Positive 


Amount of 
Intellect 

Frequencies at 1000 Bandom Periods 

A B O I> 

0 

31 

1 



1 

156 

10 



2 

313 

44 

3 


3 

313 

117 

14 

1 

4 

156 

205 

42 

5 

5 

31 

246 

92 

15 

0 


205 

153 

37 

7 


117 

197 

74 

8 


44 

197 

120 

9 


10 

153 

160 

10 


1 

92 

176 

11 



42 

160 

12 



14 

120 

13 



3 

74 

14 




37 

15 




15 

16 




5 

17 




1 

Average 

2.6 

5.0 

7.5 

10.0 
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intellect that would otherwise act, so that the more of them 
that acted the less intellect there would be, the relation 
between amount and variability would be reversed, the 
variability of a man^s intellect being as the square root of 
the amount by which the man was below the maximum intel- 
lect 1 There may be, and probably is, some combination of 
additive and inhibitive factors making the average intel- 
lects of men vary up and down from an amount typical of 
the human species ; and this may result in equal variability 
for A, who is much below the average, and B, who is much 
above it. For example, suppose there are G factors, a, b, c, 
d, e, and f, each contributing — 1, and 6 factors. A, B, C, 
D, E, and P, each contributing + 1 ; and that every intellect 
is constituted by 6 factors chosen from the 12; and that the 
momentary conditions of each intellect represent the chance 
combinations of its six factors. Then we have intellects 
whose averages range from — 3 to +3, according to 
whether they are constituted by six minus causes, or by 5 
minus and 1 plus, or by 4 minus and 2 plus, or by 3 minus 
and 3 plus, and so on. All will have the same variability, 
however, the frequencies being in the proportions 1, 6, 15, 
20, 15, 6, 1, with a mean square deviation of 1.2247. 

A consideration of the relative probabilities of various 
types of constitution of intellect out of positive and negative 
factors would be interesting, but is too speculative to be 
profitable for our present purpose. The attainment of 
greater intellect by the lack or suppression of negative 
factors as well as by the possession and use of positive 
factors is at least a possibility ; and will seem highly prob- 
able to many. 

On the whole, then, we do not need to be especially 
skeptical of the experimental findings that the variability in 
tests of a half hour from time to time is approximately 
equal over the range from, say, the ten-percentile adult in- 
tellect to the ninety-five percentile adult intellect. 
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MEASUEEMEET BY WAY OF THE FORM OF DISTRIBUTION OF 
INTELLECT IN SOME DEFINED GROUP 

If Ti, T 2 , Ta, and T*, etc., are intellectual tasks with 

^ K K + a K + a4-b K + a + b + c ^ 
which — , , , , etc., individ- 

uals of a group of individuals succeed respectively (K, a, 
b, c, etc., all being positive, K being greater than 0 and the 
largest percentage being under 100), we can measure the 
differences in difificulty for intellect between Ti, Ta, Ta, T*, 






C C 

Fig. 5. Four surfaces of frequency: A rectangle, Form A, Form C, and 

Form C reversed. 


etc., in terms of amount, if we know the form of distribu- 
tion of intellect in the group.® If, for example, n is 100, K 
is 5, and a, b, c, d, and e are each 10, the differences in diffi- 
culty will be in the proportions shown in Table 5, according 
as the form of distribution of the group is a rectangle, a 
surface like A, a surface like C, or a surface like C reversed, 
shown in Table 4 and Figure 5. 


« Our measures will approximate perfection in proportion as T„ T„ T,, T«, 
etc., depend upon all of intellect and nothing but intellect. As has been noted, 
we are assuming this for the present, reserving for full treatment later the 
influence of failures of certain tasks to utilize intellect fully, and the 
of admixture of other factors than intellect. 
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We have estimated the form of distribution for certain 
groups, using the following procedure. Choose some group 
which is caused by forces that can be studied and which, so 

TABLE 4 


Fottr Forms of Distbibitfion 


QwnUty 

Rectangle 

Form A 

Freqv^cp 

Form C 

Form C (reserved) 

K + 

84 

4 

0 

4 

K+ 1 

84 

14 

1 

1 

K+ 2 

84 

44 

114 

14 

K+ 3 

84 

9 

224 

34 

ilC + 4 

84 

15 

23 

64 

K+ 5 

84 

194 

174 

114 

k;+ 6 

84 

194 

114 

174 

K+ 7 

84 

15 

64 

2.3 

K+ 8 

84 

9 

34 

224 

K+ 9 

84 

44 

14 

114 

K + IO 

84 

14 

1 

1 

K + 11 

84 

4 

4 

0 


far as can be ascertained, represents a clustering around 
one amount of intellect with divergences therefrom due to a 
large number of causes each small in amount of influence. 

TABLE f) 

Approximate Percentages Which the Differences in Difficulty between 
Task T^, Task T^^, Task T^, etc., are of the Difference 

BETWEEN T^ AND T^, ACCORDING TO THE FORM 

OF Distribution of the Group 


For a For For For 

Bectasgle Form A Form O Form C, reversed 


T to T 

100 

244 

141 

355 

T, to T 

T 3 to 

100 

148 

105 

186 

100 

112 

91 

136 

to 

100 

104 

96 

109 

T. to T„ 

6 6 

100 

100 

100 

100 


Choose many instruments for measuring intellect (such as 
the Otis Advanced, Army Alpha, National and Terman 
tests), each of which (1) is known to correlate fairly well 
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with any reasonable criterion of intellect; (2) is different 
from the others; (3) was constructed without any depend- 
ence of the selection of elements or of the scoring system, 
upon the assumption that the distribution of intellect in the 
group in question approximates Form A. Find by each test 
by actual experiment the form of distribution for the group, 
using the scoring system for each test at its face value. 
Find the form of distribution which best fits all these vary- 
ing forms. Observe the effect (upon the form of distribu- 
tion) of reducing the chance error in the scores by obtaining 
the form of distribution for the group when two or more 
trials with the same instrument are combined for each indi- 
vidual. If the best fit distribution is of Form A, and if the 
reduction of the chance error does not produce divergence 
from this form, we may conclude that Form A represents 
closely the form of distribution of the real ability in the 
group, as measured by a scale of equal units of difference 
in that ability. The general argument is that nothing in the 
instruments themselves or their scoring favors this form of 
distribution for this group, and that it can not be due to the 
chance error, since reducing that leaves it unimpaired. 

The details of the argument and the evidence are pre- 
sented in Appendix III. They demonstrate that for Grades 
from G to 12, and probably for freshmen in colleges of equal 
standards of admission, the form of distribution of the pop- 
ulation of a grade, when perfectly' measured in respect of 
the ability required for success with standard types of intel- 
ligence tests, in truly equal units, will be unimodal, sym- 
metrical, and verj’^ closely of Form A, the ‘normaP proba- 
bility surface, the equation of whose bounding curve is the 

1 — 

exponential curve y == — == e where a is the mean 

aV2n 

square deviation. 

The critical reader should examine Appendix ITT with 
especial care. The method of measuring the intellectual 
difficulty of tasks which we adopted for our actual scale con- 
struction is based on it. It also provides support for certain 
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features of previous work which up till now has been taken 
on faith. Appendices I and II are perhaps of greater theo- 
retical importance, but Appendix III is fundamental for 
present and future practice in mental measurement. 

We can then measure the difficulty of any intellectual 
task for pupils in any one of these grades by the percent of 
the group succeeding with it, as shown in the illustration 
that follows: 

3190 pupils in grade 9 were tested with four tasks in com- 
pleting sentences. The percentages succeeding were re- 
spectively 60, 30.5, 46.1, and 37.1. We assume that these are 
intellectual tasks, that is, that success with each depends 
upon intellect. 

The form of distribution of the intellects of the group 
being Form A, a percentage correct of 60 corresxjonds to a 
division of the group at — .25330, that is, at — .25330 of the 
mean square deviation of the group (in the ability mea- 
sured in truly equal units by that task) below the average 
or median of the group (in the ability measured by that 
task) . 

.51010, -f- .09790, and + .32920 have similar meanings 
for the difficulties of tasks 21, 22, and 23. 

The differences in difficulty between the tasks are 
21-20 = .7634, 21-22 = .4122, and so on, in truly equal units, 
unity being taken arbitrarily as the mean square deviation 
of the group in intellect. 

MEASUBEMEIT'T BY WAY OF THE FOBM OF DISTKIBUTIOH OF AH 
ABBAY IH A COBBELA.TIOH TABUS 

The fourth method of attacking our problem uses, as the 
group whose form of distribution is to be determined, the 
population comprising one array in a correlation table of 
the sort shown in Table 6, where the individuals are ar- 
rayed under their scores in some examination symptomatic 
of intellect. Each array consists of two compartments rep- 
resenting the two scores (Failure and Success) attainable 
in the intellectual task whose difficulty we wish to measure. 
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For example, we might have data concerning success with 
the task in question from 1000 persons each scoring 30 in 
Army Alpha, from 1000 persons each scoring 35 in Army 
Alpha, from 1000 persons each scoring 40 in Army Alpha, 
and so on. Or we might have data concerning success with 
the task in question from 1000 persons scoring Mental Age 
8.0 in the Stanford Binet, from 1000 scoring Mental Age 
8.5 in the Stanford Binet, and so on. 

If both the total score and success in the task depend 
upon intellect, and nothing but intellect, the latter being 
one of the varying manifestations of intellect of which the 
former represents the average condition, the form of dis- 
tribution of the intellects measured in an array in such a 


TABLE 6 

Thk Correlation or StrocESS in Task 281 with Average Score in a 
Total Series or Intellectual Tasks.^ 
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the total series 







Score in 
















task 281. 

17 

18 

19 

20 

21 

22 

23 

24 

25 

26 

27 

28 

29 

30 

31 

Wrong 


44 


94 

151 213 

259 

274 

281 


75 

55 

6 



Right 

2 

4 

14 

19 

38 

60 

111 

203 

265 

302 

223 

260 

90 

39 

12 


correlation table, measured in truly equal units, will be 
symmetrical and approximately ^ normal.^ For they are 
a random sampling from the combined distribution of cer- 
tain individuals closely alike in average intellect, when all 
the variations of each individual from time to time are 
taken; and we have shown that each of these individuals* 
distributions is symmetrical and approximately “noimal.** 

The use of such an array® is in fact a convenient means 
of applying our knowledge of the form of distribution of 
the variations of an individual in intellect. It is imprac- 
ticable to obtain a hundred trials of an individual with an 

r The entries of Table 6 are genuine, but the total series is not a series 
representing all of intellect, nor is the score in it an average of many trials. 
Such data are not available. The Score in the total series’* in Table 6 is in 
fact the score in one trial of a one-half hour test of certain features of intellect. 

B We shall later see uses of other sorts of arrays. 

6 
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intellectual task; and even if we did, the results would be 
hard to interpret because of possible effects of practice. It 
is possible to find a hundred individuals who are substan- 
tially identical in their average performance at intellectual 
tasks, and test them all once with any given task. 

The measurements of the difficulty of one intellectual 
task in terms of the distance -j- or — from the average of 
one such array, expressed as a multiple of the variability 
of that array, can be made approximately commensurate 
with measurements of the difficulty of another intellectual 
task in terms of the distance + or — from the average of 
the corresponding array, expressed as a multiple of its 
variability. For we have shown that the variability of an 
individual (and so of such an array) in intellect is approxi- 
mately the same regardless of his average amount of in- 
tellect. Consequently the two multiples are of approxi- 
mately the same univ and the distance between the two aver- 
ages of overlapping arrays can be measured in terms of this 
same unit. If two arrays do not overlap, we can bridge 
the gap by inserting data from intermediate arrays which 
do form a series of overlapping arrays. 

THE DEFECTS OF THE MEASUREMENTS SO FAR DESCRIBED 

We have determined the approximate form of distribu- 
tion of a grade population, from Grade 6 to Grade 12, in re- 
spect of level of intellect at one time, if that were measured 
in truly equal units. We have done the same for a popula- 
tion (an array) characterized by identity in average of in- 
tellect measured by a random selection of times. By an ex- 
tension and refinement of the methods which we have used, 
this could be done with greater precision. 

If all that we require for the measurement of the intel- 
lectual difficulty of tasks is to secure a group of known form 
of distribution in intellect when measured in truly equal 
units, whose members we may test with the tasks in ques- 
tion, the problem is solved. Unfortunately more is required. 
Thechief defect in our procedures is that the difficulty which 
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we measure by the percentages of our group which succeed 
is not pure intellectual diflSculty. Any such task as solving 
an arithmetical problem or completing a sentence or obey- 
ing a command is deficient by not involving all of intellect, 
and often also by involving other factors than intellect. 
From the percentage of a group of known distribution in 
respect of intellect, which succeed with it, we can derive a 
close measure of its difficulty, but not of its intellectual diffi- 
culty. Although this has not been understood in the past, 
it can easily be realized by considering cases like the fol- 
lowing : A group of knowm distribution in respect of intel- 
lectual level measured in truly equal units, is tested with (a) 
leaping over a certain hurdle, (b) distinguishing a certain 
pitch from one higher, (c) spelling a certain word, (d) giv- 
ing the opposite of a certain word, and (e) giving the oppo- 
site of a certain other word. The percent of success is 
equal for a, b, c, d, and e, being, let us say, 40, so that each 
of the five tasks is -f- .2533 S.D. The five are not equal in 
intellectual difficult}’', however. Common sense tells us this ; 
and the verdict of common sense is a crude intimation of 
the scientific fact that for (a) the -f .2533 S.D. means .2533 
times the S.l). of the group in ability to leap that hurdle 
above the mean of the group in ability to leap that hurdle, 
vrhereas for (d), the + .2533 S.D. means .2533 times the S.D. 
of the group in ability to think of the first opposite above 
the moan of the group’s ability to think of that opposite. 
Ability to think of the second opposite may conceivably 
differ from ability to think of the first opposite by involving 
much more of intellect, or much less of non-intellect, or both, 
in the same way that the ability to think of the first opposite 
differs from the ability to leap a hurdle. If we take the 
tasks chosen as intellectual tasks and put in any of the stock 
intelligence examinations, they will so differ. This has been 
abundantly proven by investigations which will be reported 
in Chapter IV. Moreover, no one of them will measure all 
of intellect and nothing but intellect. 

In fact, no one short task does or can involve all of intel- 
lect and nothing but intellect. Any one short task measures 
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only a fraction of intellect and is influenced by other forces 
than intellect. That is, any one short task measures intel- 
lect plus an error. The nature and amount of this error 
must be considered in connection with any procedure for 
estimating- the intellectual difficulty of a task from the per- 
centage of individuals who succeed with it.® 

There are other hidden assumptions and weak or even 
missing links in the argument by which we proceed from 
knowledge of who and how many can do a task, to a meas- 
ure of its intellectual difficulty. In the next chapter we shall 
expose these, subject the entire argument to a much more 
rigorous treatment, and seek to remedy the defect noted 
above and such others as are found. 

® The exposure of tbis defect should not diminish our use of the general 
procedure of inferring degree of difSculty from percentage of failures in a dis- 
tribution of known form. On the contrary, now that we are aware of the defect, 
we can make much better use of the procedure than when we were ignorant of it. 
As we shall elsewhere show in detail, if we replace a single task by a composite 
of forty tasks, and use twenty or more right as our mark of * * success, ^ ’ we can 
use the procedure with better results than have ever been obtained hitherto. 



CHAPTER ni 


The Measurements of the Intellectual Difficulty op 
Tasks and of Level of Intellect : More 
Rigorous and Exact Methods 

In the two previous chapters we have operated with pro- 
visional and somewhat vague definitions and inexact as- 
sumptions, largely in order to maintain continuity with 
what has been done to date in the measurement of intellect- 
It is now necessary to treat the whole matter of intellectual 
difficulty and level of intellect more rigorously. 

We have assumed (1) that there is such a quality or 
characteristic of man as altitude or level of intellect; (2) 
whose amount or degree is measured by the height at which 
it can attain success with a series of intellectual tasks 
ranked for difficulty; (3) that the same individual differs 
in the amount or degree of it which he has available from 
time to time; and (4) that different individuals differ in 
the amounts or degrees of it which they have available on 
the average. (5) We have defined intellectual tasks only 
loosely and vaguely as those in which men esteemed very 
intelligent differ most from men esteemed very unintelli- 
gent. (6) We have defined intellectual difficulty only 
loosely and vaguely as that characteristic of a task, an in- 
crease in which reduces the number of intellects who can 
succeed with it, eliminating those esteemed unintelligent 
more rapidly than those esteemed intelligent. 

Since we are treating intellect as the ability to perform 
intellectual tasks, our primary need is a clearer and more 
exact notion of intellectual tasks. We can reach this in 
either of two ways. The first is by assuming that certain 
abilities, such as to understand directions, or to know facts, 
or to use relations of likeness, part and whole, actor and 
acted upon, genus and species, and the like, or to use facts 

69 



60 


THE MEASUBEaCEITT OF INTEIXIQENCB 


together, and certain tasks which represent them, are as a 
whole iulelloctual. We musi then describe these tasks, and 
the credit or weight to be attached to each, precisely, and 
put them in a total series in such form that an individual 
intellect can attempt them all. 

The second is by assuming that the ranking of individ- 
uals in an order from idiots to Aristotles for amount of in- 
tellect’ by some defined consensus of opinion is valid. We 
must then describe this consensus and the method of its 
operation. 

If we take the former way, we may attach the term 
‘‘intellectual task” to any selection from the total series 
which, when treated in the same way, gives measures for in- 
dividuals which correlate 1.00 with measures from the 
total series. This task would be just as intellectual as the 
total series, would involve just the same abilities as it in- 
volved and no others. Kvery thing would be rigorous and 
precise after the selection of the tasks and arrangement for 
scoring them. In practice a selection which gave a correla- 
tion slightly under 1.00 might be accepted as substantially 
an intellectual task. 

Further, if any selection from the total series, when 
treated in the same way as the total series, correlates as 
closely with the total series as its own self correlation per- 
mits, that selection is an intellectual task.® Its failure of 

3. Amount of intellect means hero the average amount which the individual 
has available over a period long enough to be representative of him. 

2 Let the group be measured a number of times by the total series 1 and 
by the task t. Let be the correlation between any two measures by i and 
let Ttjtg be the correlation between any two measures by t. Let be the 

correlation between any measure by t and any measure by i. Let rtodiin be the 
correlation between the average measure by t repeated indeftnitely, and i re- 
peated indcdnitely. 

Then by the well-known attenuation formula, 

^taoloo 

and if rt j is no less than the geometric mean of rt,to . will be 

1.00 or perfect correlation. Since in a group of wide range in intellect will 
approximate unity, we may set as the requirement that, in a group of wide range 
in intellect, should be little if any less than Vrt t • 

3> 1 i 2 
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perfect correlation is due entirely to the fact that the intel- 
lect of the same individual varies from time to time, not to 
any intrinsic inadequacy or irrelevance. 

If we take the latter w^y, we may attach the term “intel- 
lectual task^' to any task or collection of tasks, the score in 
which correlates -f- 1.00 with the ranking of individuals in 
an order for intellect by the consensus, or correlates as 
closely with that ranking as its own self-correlation permits. 
By collecting such tasks we may obtain a total series which 
may then be used as a criterion in the same manner as a 
series derived by the other method. 

These two procedures are more definite and systematic 
and rigorous forms of what has been done in test construc- 
tion. Psychologists have on the one hand taken tasks which 
they assumed to be intellectual and have put samplings of 
them into teams of tasks. On the other hand they have as- 
smned that a ceitain validity attached to rankings by 
teachers, by the sifting process of advancement in school, 
or by other forms of consensus, and have selected those col- 
lections of tasks which showed high correlations with such 
a criterion. 

What abilities and tasks shall be treated as intellectual 
is essentially a matter of arbitrary assumption or choice at 
the outset, either directly, of the abilities or tasks them- 
selves, or indirectly, of the consensus which provides the 
criterion. After the first choice is made, tasks not included 
in it, and even not known, may be found to correlate per- 
fectly with the adopted total, and so to be “intellectuaP’; 
but their intellectualness is tested by and depends on the 
first arbitrary choice. Had a different first choice been 
made, they might not be intellectual. This arbitrariness is 
a sign of weakness, but it is at present unavoidable. We 
have to define intellectual tasks as best we can, and trust 
that future scientific uses of the definition will improve it. 
We shall see later that the arbitrariness is greatly tempered 
by certain guiding principles and facts, and that a total 
series of intellectual tasks can be defined so as to represent 
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a fairly clear, tinified, coherent feature of human life, suit- 
able for theoretical treatment and of great practical im- 
portance. 

INTELLECTUAL DIFFICULTY 

We are now in a position to make the concept of intel- 
lectual difficulty more rigorous and exact. Having, by 
either method, derived a series of tasks (N in number) 
which as a whole are intellectual, we define the group in 
respect of ■whose members the difficulty of the task is to be 
determined, (for example, as ten thousand taken at random 
from all living human beings twenty years of age), test each 
individual of the group with each of the N tasks, rank the 
tasks in order by the percents succeeding "with each, and 
divide them into x sub-series® (called D„ Da, Da, etc.) in 
accord with the ranking, Di containing the “ easiest,’* Da 
the next “easiest,” and so on. 

If the score in each of these sub-series of tasks gives cor- 
relations of 1.00 (or as high as its self -correlation permits) 
with the total series, we can define intellectual difSculty as 
that feature, which Di, Da, Ds, etc., have in increasing 
amounts. They differ in nothing else of consequence to our 
inquiry, the score in each being determined by all of the 
intellect defined by our total series and nothing but that 
intellect. 

The attainment of such sub-series may conceivably be 
an impossibility. It may be that, no matter how large N 
(and consequently N/a;) is, the sub-series of tasks at some 
points in the success-frequency ranking may fail of perfect 
correlation with the total series. The kind of tasks chosen 
as intellectual may, for example, vary in such manner that 
all of even a “small” intellect can not be utilized without 
tasks from the very hard end ; or in such manner that non- 
intellectual factors can not be eliminated or equalized for 
all twenty-year-old individuals without tasks at widely sep- 

s For eonvaniance of exposition we will assume that the number of taAs 
in each sub-series is the same, though the argument will hold regardless of the 
size of the Bub-eeriee» 
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arated points of the success-frequency ranking. This is a 
matter for experimental determination after the total series 
has been chosen and the group of intellects in respect of 
which difficulty has been defined has been chosen. The re- 
sults of such experiments are of great significance, inform- 
ing us of the degree to which amounts of intellect as defined 
by the total series do represent increases in the same kind 
of thing, and are amounts of some unified, coherent fact in 
nature which can properly be isolated in thought from non- 
intellectual factors. Our experiments on this matter will 
be reported elsewhere,* but we may note now that they indi- 
cate that intellect has a rather high degree of unity and 
consistency and independence of non-intellectual factors; 
and consequently permits a fairly close approximation to 
sub-series of tasks which, as total sub-series, do approxi- 
mate to perfect ‘‘intellectualness,” while differing enor- 
mously in difficulty. We proceeded by an approximation 
to the first method, and later checked our choice by an ap- 
proximation to the second method. In our choice we were 
guided by the following considerations : 

(1) Of psychological theory: — (a) that responding to parts 
or elements or aspects of situations is more “intellec- 
tual” than responding to gross total situations; (b) 
that responding to parts or elements or aspects which 
do not present themselves separately to sense but must 
be abstracted is more intellectual than responding to 
those which do; (c) that responding to relations be- 
tween objects is more intellectual than responding to 
objects; (d) that, in particular, responding to so-called 
subjective or logical relations, such as likeness and dif- 
ference, is more intellectual than responding to the so- 
called objective relations of space and time; (e) that 
organizing several mental connections or habits to se- 
cure a certain result, “thinking things together,” as 
James put it, is more intellectual than using. one habit 

* In Appendix IV and Appendix V. 
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at a time; (f) that responses to novel sitnations are 
likely to be more “intellectuaP’ than responses to 
familiar situations. 

(2) Of the theory of measurement: — (a) that the tasks 
representing any one ability should be capable of very 
fine gradation from very easy to very hard; (b) that 
they should be capable of very wide extension by alter- 
nates at any degree of difficulty; (c) that, so far as pos- 
sible, any one ability should represent in some real and 
useful sense something varying only in amount, so that 
the different degrees of it might properly be repre- 
sented by numbers. 

(3) Of common sense: — (a) that the tasks should be from 
among those which had high standing on the basis of 
correlations with reasonable criteria; (b) that they 
should be convenient for use in the actual measurement 
of intellect; (c) that they should be tasks concerning 
which subjects for experiment were obtainable. 

Over and above the narrowness due to these considera- 
tions, our choice is also deliberately narrow. We have not 
included any tasks involving responses to actual human be- 
ings or to material objects present to sense — tasks of what 
has been called social intelligence and mechanical intelli- 
gence. Our tasks all concern responses to ideas and sym- 
bols, especially words and numbers. The reasons for this 
need not detain us here. Also we limit ourselves to tasks 
which are intellectual for a group of persons bred in the 
United States and aged twelve or over.® The reasons for 
this narrowness may also in general be omitted.® 

5 The tasks will very probably serve to measure intellect for younger ages 
even more accurately than tasks now in use, but we have not demonstrated this 
to be BO, 

® The chief reason was that the measurement of intellect in children up to 
twelve or fourteen and the doimition of the measurement by an age-scale are 
in a far more satisfactory condition than the measurement of intellect at older 
ages and at higher levels. 
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IITTBLJLECT cavd 

In view of these elaborate and arbitrary restrictions, the 
intellectualness of our total inventory of tasks, and the 
intellect whose level or altitude, range or width, and facility 
or quickness it measures, will be called hereafter Intellec- 
tualness CAVD and Intellect CAVD (the symbol CAVD 
refers to the four series of tasks which constitute it — com- 
pletions, arithmetical problems, vocabulary and directions). 
The total series of tasks concerns four lines of ability 

C- To supply words so as to make a 
statement true and sensible. 

A. To solve arithmetical problems. 

V. To understand single words. 

D. To understand connected dis- 
course as in oral directions or 
paragraph reading. 

The arrangement of scoring is such as to attach equal 
weight to each of these four varieties of tasks. 

The whole series is put into a rough approximate order 
of intellectual difficulty by the methods described on pages 
39 to 56 of the previous chapter. Consequently all the 
single tasks or task elements of any one sub-series are of 
somewhere nearly equal intellectual difficulty. 

Kach single task is scored 1 (right) or 0 (wrong or 
omitted). The nmnber right at each level, that is in each 
sub-series, is recorded. The time required for each task 
may be recorded, if desired. Selections of forty single- task 
elements from each of certain sub-series of the total series 
are shown below, maldng composite tasks A, B, C, D, N, O, 
P, and Q. 

TWe shall sometimes use also Intellect CAVDI, which is constituted by 
including a dfth sort of task — ^to understand and answer questions which re- 
quiio information about such facts as are considered by the world to-day worthy 
of study in school and of record in encyclopedias; plus organization thereof 
and sagacious inference therefrom. 

I 37, 38 a- 
<»n8ions. 
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SUB-SEBIES A 

Sentence Completion, Oral A 

1. You are sitting on a 

2. We take a ride on the 

3. At night you sleep in 

4. You like to drink 

5. We get up in the 

6. Mary has a ring on her 

7. You wear gloves on your 

8. The snow is on the 

9. We go to church on 

10. You wear a on your head. 

arithmetic, oral a 

13. Counts 2 pennies. (Binet procedure, hut credit for 
success 2 of 3 trials.) 

12. Counts 3 pennies. (Credit if successful in 2 of 3 
trials.) 

13. “Show me 2 pennies.*^ (Credit if successful in 2 of 
3 trials.) 

14. “Show me 2 pennies.” (Credit if successful in 3 of 
3 trials.) 

15. Recognizes 2 fingers. (Credit if successful in 3 of 5 
trials. ) 

16. “Show me the littlest pencil; show me the littlest 
one of all,” showing 3. (Credit if successful in 2 of 
3 trials.) 

17. “Show me the littlest square; show me the littlest 
one of all,” showing 3. (Credit if successful in 2 of 
3 trials.) 

18. “Show me the biggest square; show me the biggest 
one of all,” showing 3. 

19. Adds unseen, 1 plus l.“ (Credit if successful in 2 of 
3 trials.) 

20. Subtracts unseen, 2 minus 1.® (Credit if successful 
in 2 of 3 trials.) 

■ In adding ur* as 2 plus 1, for instance, the procedure is as follows: 

2 pennies are s abject answers (rightly or wrongly) the question. 
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SUB-SEBIES C 
Senteistce CoMpusTioia-, Oral. C 

d. Clouds are in the 

IB. We send children to school, because they 
mufet 

t . We burn in the stove. 

. The is barking at the cat. 

5. We wash clothes with and water. 

j6. Grass is 

*7. is sweet. 

8. We see with our 

9. Boses and daisies are 

10. The eats the mouse. 

>< X 



Fio. 7. The pictures used with Directions Oral B, 37, 38 and 39: rediiced to 

one-half the original dimensions. 


7 
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Arithmetic, Oral. C 

11. Counts 5 pennies. (Credit if successful in 3 of 3 
trials.) 

12. Counts 10 pennies. (Credit if successful in 2 of 3 
trials.) 

13. "‘Show me 3 pennies. (Credit if successful in 2 of 
3 trials.) 

14. “Which is the bif^gest pile?” showing 10 and 5 pen- 
nies. (Credit if successful in 3 of 3 trials.) 

15. “Two and one make ” (Add “what?” 

if necessary. 

16. Recognizes 3 fingers. (Credit if successful in 4 of 5 
trials.) 

17. “Which is the biggest, a chair or a cup?” (Credit 
if successful in 2 of 3 trials.) 

18. Subtracts unseen, 5 minus 4. (Credit if successful 
in 2 of 3 trials.) 

19. Subtracts unseen, 3 minus 3. (Credit if successful 
in 2 of 3 trials.) 

20. Subtracts unseen, 2 minus 2. (Credit if successful 
in 2 of 3 trials.) 


Vocabulary C 

The method is as before. The words used are : 

21. camera 26. pistol 

22. stationery 27. vase 

23. hole 28. stamps 

24. corn 29. tiger 

25. puppy 30. kennel 


Three of the rows of pictures used in this type of test 
are shown in I’igures 8, 9 and 10. It will bo observed that 
the task sometimes involves a considerable degree of ability 
in interpreting the pictures. 





Pig. 8. Picture used with ‘'lanop. 
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FiO- 10. Picture used with ‘*6ork.'* 


Directions, Oral C 

In the actual tasks the drawings have twice the dimen- 
sions of those shown here : 

31. ‘‘See the square (A li inch square is shown at 
the top of a sheet 11 by 8^.) 

‘ ‘ ]Vlake a ring in the square. ’ * 

32. “Now make another ring in the square.’’ 

33. “See the ring? Make a cross in the ring.” (A 
circle 2 inches in diameter is shown near the middle 
of .the sheet.) 

34. “See thecup. Draw a line around the cup.” (Fig. 
11 is shown at the bottom of the sheet.) 

33. “IVlake a ring and a cross up here,” pointing. 

36. “Make a cross where the line is.” (A line 2^ 
inches long is shown, parallel with the bottom of the 
sheet.) 

37. “Draw a line to finish the square.” (A half-inch 
square with the left-hand side omitted is showm.) 

38. “Make a cross in here,” pointing to a triangle whicdi 
is printed with a square on one side of it and a 
circle on the other. The square is 1^^ in.; the tri- 
angle has a base of 1^ in.; the circle has a diameter 
of li in. 

39. “Make a cross X m the square.” (Fig. 12 is 
shown.) 
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40. “Make two squares out of these.” (Two f in. 
squares are shown, one with the right-hand side 
lacking, the other with the lower side lacking.) 



Fig. 13. 


^ ^ HZ 

Fig. 14. 


SUB-SERIES D 

Sentence Completion, Oral D 
There being only eight tasks, each is counted as 1^. 

1. Boys baseball. 

(“Playing” and “play ball” are called wrong.) 

2. The stars and the will shine tonight 

3. Two and one make 

4. A boy has and legs. 

5. The bird sings ; the barks. 

6. Men are than boys. 

7. The pulls the cart. 

8. Horses are big and 
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Abithmbtic, Oral D 

11. Counts 15 pennies. (Credit if successful in 3 of 3 
trials.) 

12. Recognizes 4 fingers. (Credit if successful in 3 of 5 
trials.) 

13. Show me 4 pennies.’’ (Credit if successful in 3 of 3 
trials.) 

14. “How many fingers have you on one hand?” 

15. Recognizes 3 fingers. (Credit if successful in 5 of 5 
trials.) 

16. Recognizes 5 fingers. (Credit if successful in 5 of 5 
trials.) 

17. “Which is biggest, 3 or 1?” (Credit if successful 
in 2 of 3 trials.) 

18. Adds unseen, 2 plus 2. (Credit if successful in 2 of 
3 trials.) 

19. Adds unseen, 3 plus 2. (Credit if successful in 2 of 
3 trials.) 

20. Subtracts unseen, 5 minus 3. (Credit if successful 
in 2 of 3 trials.) 


Vocabulary D 

The method is as heretofore. The w^ords are: 


21. tools 

22. fuel 

23. screw 

24. angel 

25. cartridge 


26. trumpet 

27. cube 

28. cork 

29. blade 

30. arrow 


Directioits, Oral D 

The illustrations shown here all have dimensions half 
sed in the actual tasks. Each row is also in the 
isks separated from the one above and from the one 
[by from 1 to 3 inches. 

i 

‘Make a cross inside the little square.” (Fig. 13 is 
ihown.) 
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32. *^Draw a line to make this a cross,” pointing. (A 
thick line f in. long, parallel to the side of the sheet 
is shown.) 

33. “Make a ring on the cup.” (Fig- 11 is shown.) 

34. “See the ring. Make 2 crosses in the ring.” (A 
circle 2 inches in diameter is shown.) 

35. “Make a cross on top of the boy’s head.” (Fig. 13 
is shown, on a new sheet.) 

36. *^Draw a line around the big hand.” (Fig. 14 is 
shown.) 

37. “Make a cross on the horse.” (Fig. 12 is shown, on 
a new sheet.) 

38. “Make a cross outside the big square.” (A second 
coi)y of Fig. 13 is shown.) 

39. “JSfake this a circle,” pointing. (An incomplete 
circle with a diameter of f inch, lacking the right- 
hand quarter, is shown.) 

40. ‘'Make a line outside the ring.” (A circle 2 inches 
in diameter is shown.) 

The sub-series N, O, P, and Q which follow presuppose 
ability to read in the individuals measured by them. 

SUB-SERIES N 
Sentence Completion 

Write words on the dotted lines so as to make the whole 
sentence true and sensible. Write one word on each inch 


of dots. 

1. At time was progress 

rapid during the last half of the nine- 
teenth 

2. He will come to the meeting ! 

the fact 

he ^ rather stay quietly at home. 

3. His friends, wished to dissuade him 


from this undertaking, asserted that 

he follo^^ — ‘r advice would with- 

dro""-^!^ are big anv^t. 
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It woTild several pages^ * to 

contain the list. ^ 

Standing beside the grave. 

great Englishman ^.1 

enough for ns to know ^ 

lived and died, and made the his 

heirs. 

You may safely conclude that you : in 

yourself the means of at the 

the fact that you disagree wi^|j|^ 

shal^continue to aid you. 

At ancient banquets the the chutor' 


seems 


o A r the treasure he had come to seelfe.» 

9. A«*-4 — in his owf? 

probably it existed 

— ::: affirms V 

. -1? I.!.. 1 




affirms V 

all men with certain 1 




inalienable — 


T 

Arithmetic 

11. A camp has food enough to last 300 men 4 months. 

How long will it last ^00 men 1 a + n "P tvi si 

12. A wah.-h was set correct at noon 

■ on Thursdax it was lo seconds fast At that rate,^ 

how much will it gam in half an hour . ^ 

13 *^^ave sixths equal how many thirds! » Vt«lf ? 

14 Hdw many quarters of a quarter equal of a tolf 
it iW it take a man to walk 14 n^es at the-" 

rate of 3 miles an honr while wa^, if he makes 
Sree stops of an average length oflOmmutesra^t 

‘.1 £o 




ply the mii^siiiff nuinii 

S 1 H cost d 

.^'li., -.3 X costs ,-'as mucli ap 2'‘ J. , ;r4&^ 

■ l-J ■ “ i 1 IX -'fel 

.. 8 4 ^::' 1 

o(i' .- 4 . 0 .'- m 

‘ ,^w^, .V' . ,^. v»» 

. I • ,-. / , M. .-v'' 

Look at ^ret word i» life^^Ulls^oV 'wld fa^ 
lie liae whwm means tke 

Wiow the wayTo . 

• one number for each line. 

.J. bv.of’t ;;l afraid.....2 wti^iXs 8 large 4 aiiimal._..5 bte®-- ' ■■‘'^£|<^^^^&''.' ' 

£. Ijby 1 cradle... .2 xnother......3 mtte child... 4 yoiith.„/!^'g*fr 

a rflia.? - 1 lift up .„..2 dr.'ig.....3 sun... i bread 5 deluge ■ / 

'• ^■"*‘‘'* ^ 2 cannot; see 3 game... 4 unhfippy...„ 6 . ^^r^',,;;' i 

ext..n 1 cube... .2 janitor—S compas 8......4 «R*.bj«hop:_..C -i^'Shgers 

>uck]er 1 keel 2 eervant-.-S stag. 4 sliieltt„,..-S^ seraper' 

«iimostt.y I hatred ^anJanation disobediouce. 4 diversity..;.® /^dahin 

ouflagr.ai..on 1 canuv^.... 2 celebration . .3 decoration with 

Mifidentiax 1 respectable......! -^re 3 sensitive . 4 J 

«.cr.vener ;^mr|Aer..... 2 foi 5 r .„3 ehaplain..„. 4 ' 

n '■ b<--‘'k--4 slairf|rdr.;ijF'bdtiffe-VV--'^ 

UM t.te 1 po^te.„.'r^^ .3 promineittI 4 |trail 5 ' '- 

'^jiibinaraii^y^ 

^ .1 j- 

2 

i. 
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31. Show me the man you honor. I know by that 

symptom, better than any other, what you are 
yourself. — (Carlyle. ) 

A man is known by the company he keeps. 

Tell me what youVe done and I will tell you 

what you are. 

A man is known by his idols. 

Show me your ehijjs and I will tell you whether 

you are a f^ood woodsman. 

32. It is one thing to see that a line is crooked and 

another thing to be able to draw a straight one. 

It is one thing to see the mote in our neighbor’s 

eye and another to see the beam in our own. 

Those who see mistakes cannot always correct 

them. 

As the eye is trained to accuracy the hand de- 
velops skill. 

.AVo may recognize faults that we are unable to 

overcome. 

33. If we agr« that morality is what is social and im- 

morality, anti-social, we shall be led to inquire 
of any course- of action how it affects the welfare 
of society. — (P§aison.) 

/..To judge.^s^ther an action is good, we must 


investigate its results on society. 
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♦ 

THE ]ME\SniEMENT OF IKTELLIGEJSTCE 

Tie iin])li<*p that doctors may make mistakes 
■which rt'inain iiiKlot<*cted. 

35. Tt is easy to be virtuous when one’s own conveni- 
ence is not affected.- (Steven son.) 

Virtue it.s own reward. 

It is easier to |u<'acli virtue than to ]n'xietice it. 

It i-^ diliicult to d(> riiiiit wluui it conflicts Avith 
our in<'1inatif>nt>. 

W<‘ would all h<‘ Airtuoir^ irA'irlue were merely 
a matter of doinii, what A\e enjo\. 

36. Don’t cross the bridge before you come to it. 

Ijook lK*l‘or<‘ A oil l('a]>. 

Don’t horiow trouhle. 

Don’t lock the liarn al'tt'r tlie horse is oone. 

I’ake can* of todaA and l(*morrow' a\ ill take care 
ol* itself. 

Head fhiif and I hen in tie tin Jiiad it again if 

you need to. 

There is an old sa\ iiui. “As harmless as a fly;’’ and 
until r(*cenfly the tly has he<*n r(*ivardo<l onI;v as an un[)leas- 
aiit hut haimh'ss nuisance. Had our t’oi‘<‘l’ath«*rs knoAvn as 
much about flies as A\e innv know, the\ niit^ht have made the 
proverb, “As <hini'<‘rous as tlu* fly.” His oriaiu and his 
hahits are of tlie worst sort. 

37 . Copy the AAords A\}iieh mean the* same as jiroA'erb. 

7? aa ./os and then u rite the anf,ut^,. Head if again if 
yon need to. 

KVEKY HOME NEEDS A GARDEN 

A MAGAZINE pubiished to promote r<“al gardening. 
Most peo])h' <lo not Ihink nuK'h about their gardens at this 
time ol* the A(‘ar, nut if more ]»eoph* diil, tli(*re avou1 (^ be 
more good gardei 
at a jiremium, wi 
how to grow" do'll 
and have a gard< ii a *1 
seeing things gio'^v 
you need the X-Y.ii^ 
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38. At what time of year do you think this advertise- 
ment appeared? 

Read this and then write the answers. Read it again, if 
you need to. 

However certain it may seem to ho that men work only 
boeanso tJiey iiuist, and would avoid labor except for the 
food, clothing and luxuri(‘s tliat are its rewards, the facts 
may well bo to the cotilrary. It can hardly bo the case that 
men dislike* work because* they wish te> Ik* utteily idle. Kor 
mere rest, more inactivity, is not commonly enjoyed. To 
have iiothing to do is not what men seek. Were that so, we 
should envy the* prison(*r sliut uj) in his cell. If men had 
to choose between a life spent at eight hours of weirk daily 
in a factory and a life s])ent at eiglil hours of sitting on a 
throne* wilhout iiun ing hand ejr foe)i, many of them wenild, 
after trying be)tl 2 , choose tin* fomier. Adivity of body or 
mind, at which a man can succeed, is, in and of itself, rather 
enjoyed than disliked. 

39. What, acce)reling to the jiaragraph, has no appeal 

])er scf 

40. AWiat is it the author of tlee paragraj)!! suspects men 

might che)ose unh'ss they had tried it? 


1 . 


2 . 

3. 

4. 


STTB-SERIES O 

Ce)MPLETIONS 


India is rich in t>f scenery and climate, 

the mountains to vast 

deltas raised a few 

above sea 

Undue consciousness often the flow of 

expression — diffuseness is detrimental 

to a clear and — our iueas. 

Knighthood and Ciliv 
axe 


svnoh 

Throughout the river Mf 
harvests, and, -w. 


..are 
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5. a man time sufficient for 

all laudable pursuits, and sufficient for 

all g’enerous purposes, ho is free 

shadow of blame or reproach. 

6. Maize contains small a proportion of 

nutritious matter it 

not for horses which 

fast woi k is 

7. The drafting a measure depends 

the pains and skill ex- 
erted by its 

8. is natural that b(‘ing dissfitisfied with 

the , we should form a too 

estimate of the past. 

9. He believed in liard things 

because hard. 

10. Not — do living things grow themselves, 

- produce life 

xr lik f 


\V 


h. 


Arithmetic 


11 


IS 


12 . 


A factt earns $70 a day for its owner when it ii 
working i “opacity and $ir) a day when it is work 
ing to half capi"v^*ty. In how many days will it earn 
$1,000 if two da.ys of every three are only half 

capacity? - j tt 

A company marched 120 miles in o days. How many 

times as fast must 'they march to cover 90 miles in 

A^man started with $.^2,500 and doubled his capital 
^ for hytJ mucfi liad he at the 

(>0^. liTOe 

% of Oap 

trniw at »0 .Hr'w ”g_wr'- 
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16. How long will enough food for 400 men for 120 days 

last 250 men? 

17. How many times as big an area has a circle with radius 

of 9 as a circle with radius of 3? 

18. At an average rate of 12 miles per hour for the first 

half of the time spent and 10 miles per hour for the 
last half of the time spent, how long will it take a 
truck to cover 110 miles? 

19. Three fourths equals how many thirds? 

20. A push-cart man hnj’s eggs at Ibc. per dozen and sells 

them at 15 for 25c. How many eggs must he sell to 
gain $1.80? 


Vocabulary 

The direetioiis and samples are the same as in Sub- 
Series N. 


23. gainsay 

22. oc'logue 

23. cloistered 

24. recjj)»’o(‘al 

25. accolade 

26. benighted 

27. madrigal 

28. pinnace 

29. broach 

30. ncctaiine 


1 persuade 2 beslircw 3 deny 4 profit 5 imprint 
1 obi tuary 2 a poem 3 carousal 4 ^^pigram 5 portrait 
1 miniature 2 buncbeJ 3 arched 4 "maTa^' fi secluded 
1 baturinne 2 mutual 3 receptive 4 morose 5 careless 
1 salutation 2 anchovy 3 procession 4 bivouac 5 acolyte 
1 fraudulent 2 weary 3 insuperable 4 ignorant 5 venal 
1 song 2 nionlel>aiik 3 lunatic 4 ribald o sycophant 
1 a boat 2 doublet 3 pinnacle 4 hold fast 3 forfeiture 
1 dodge 2 clasp 3 open 4 tt>p 5 edify 
1 bouillon 2 a fruit 3 a jewel 4 a drink 5 diurnal 


Diiections and Comprehension of Sentences 

AND I^lRAGRAPHS 

, > M only ^ tasks hi pf the usual te^. Sm 

each as ^ 

In each set of sentences, checfz fhe two tvhich mean most 
nearly the same as the sentence printed in heavy type. 

3Z. Bciler be a big'frog ia a little puddle than a tadpole 
in' a'teJ^. 

^iHettor the head of an ass than the .ail of a 

horse. 

J[ had rather be a 'door-keeper in the house of 

^my God than to dwell in the tent 4 of iRioked- 
^jiiess. '■ ' 
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Better to reifrn in hell than servo in heaven. 

Better to ho a beggar in Borne than a prince 

in a village. 

32. Don't cross the bridge before you come to it. 

[jook before you leap. 

Don’t borrow trouble. 

Don’t look the barn after the horse is gone. 

Take eare of today and tomorrow will take 

care of itself. 

The paragraj)li for questions 33 and 34 is the last para- 
graph in Sub-Series N. 

33. AVhat ehoiee is described as an argunienl that work, 

merely as such, is not always avoided? 

34. In what respect is a ]>risoner in his cell like a man with 

a million dollars? 

jRead this mid then iri ite the answers. Read it again if 
you need to. 

TITK AJVIKBICAX STATE 

lie who looks at a ma]) of tlie Union will be struck by 
the fact that so many of tin* Ixnuulary lines of the Stat<*s 
are straight lilies. Those lin(‘S tell the sairn^ tale as the 
geometrical plans of citii-s like Ijeuingrad or ^Vashington, 
where eveiy streid runs at the same angle to every oilier. 
The States are not ar(‘as set off by nature. Their boun- 
daries are for the most part not natural boundaries fixed 
by mountain rang<‘S, nor evmi historical boundaries due to 
a seri(‘s of events, but boundaries, purely artificial, deter- 
mined by an authority which carved tlie national territory 
into strijis of convenient size, as a building company lays 
out its suburban lots. Of the States subsequent to tin* origi- 
nal thirteen, California is the only one with a genuine nat- 
ural frontier, finding it in the chain of the Sii^rra Nevada 
on the east and the Pacific Ocean on the west. No one of 
these later States can be regarded as a naturally developed 
political organism. They are as trees planted by the for- 
ester, not self-grown with the help of the seed-scattering 
wind. 
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36. To what may we attribute the similarity between the 
plans of certain cities and the arrangement of the 
States? 


SUB-SERIES P 


Completions 


1. The monuments of P e r s e p o 1 i s 

the use incense 

as in ancient Persia 

as Babylonia. 

2. Ever since the hearing before him the 

governor g i v i n g 

spare moin(‘nt a 

of the east*. 


3. So far the displeasure of 

tin* people by the w'll of their repre- 

s<*nlatives, a l*r(*sident gein'i'ally gains 

by the bold use of his veto powi'r. It conveys the 
liiTnn(*ss; it show^s 

- has a view an<l does 

- - — to give effect to it. 

4. The of charact(*r is its ability to 

liberty from license. 


5. 


6 . 


Judicial decisions are <d‘ or less author- 
ity as ]'rec('dents to ('ircumstances. 

The deepest dirfer<‘nce, practically, in the moral 

of - is the difference 

the easy-going and the 


mood. 


7. 


Ibsen’s whole problem 

slated, is the ... 


8)f llriiP sixt^lth ceuTury, ~ 


i it has .yrjslii been 

Irpvidwil to 



"was ^ot more 


decidedly the land of the fine arts, 

waa^not more — the land of Jnrfd' th«o- 

td^cal spe^atten, Spain was the 

of statesmen and ' * 
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9. The human race may be as parcelled 

into ^ a of distinct 

groups societies, 

greatly in size and circumstances. 

10. Farmers brought up in the traditions of the 

of New England, on going where close 

association and cooperation were to 

carry on irrigated agriculture, found that it took a 

long and involved 

waste to learn to act 


Akithmetic 

s Write the numbers and signs in each line below in the 
proper order, so that they make a true equation as shown 
in the three sample lines. Use the bottom of the page to 
figure on if you need to. 

3 3 6 = + 3-]-3 — 6 

Sample lines 4 7 8 20 — + X * ‘ 7y^4. = 20 8 

12 3 3 7 18 = + — X ( ) 7^2 = 18— (3y^3) 

11. 1 3 3 3 3 21 = + — X-^-O 

12. i 2 3 5 33 = + XX() 

13. li 2 2 2 8 12 = + XXX() 

14. 2 2 5 10 70 = +X-^() 

15. 3 1 4 4 20 = — XXO 

A 2 for 5c. E 3 for 10c. J 3 for 25c. 

B 31c. per lb. F 4 for 10c. K 4 for 25c. 

C 41c. per lb. G 50c. per lb. L 6 for 25c. 

D 3 for $1.00. H 21c. each. M 6 for $1.00. 

A, B, C, D, etc., are articles costing as shown above. lA 
means 1 of A, 2A means 2 of A, 3 A means 3 of A, etc. Sup- 
ply the missing numbers in lines 16 to 20 as shown in lines 
I, III, and III. Use the bottom of the page or another sheet 
of paper to figure on. 

1. 2 A cost 1 as much as 3 E. 

'll. 1 E costs 11 as much as 1 F. 

II^. 3 D cost just as much as 12 J. 
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16. 

3 F costs 

as much 

as 2 J. 

17. 

3 K “ „ 

ii 

5 L. 

18. 

2 K 

ii 

1 dozen H. 

19. 

2JH-1M“ __ 

ii 

10 H. 

20. 

3 A _ 

ii 

1 K. 


VOCABULAEY 

Directions and Samples as in Sub-Series N. 

21* monomania 1 flying machine 2 conceit 3 one-colored 4 endogen 5 aberration 
22* saturnalian 1 reptilian 2 impertinent 3 gloomy 4 impregnated 5 riotous 

23. pristine , 1 flashing 2 earlier 3 primeval 4 bound 5 green 

24. quaterni(^n 1 oflBeer 2 fourfold system 3 four-line stanza 4 tremolo 5 geologic age 

25. predatoi|y 1 hasty 2 ante dated 3 rapacious 4 foretold 5 four-footed 

26. persiflage 1 camouflage 2 wit 3 banter 4 vivacity 5 metaphor 

27. encomium 1 repetition 2 friend 3 pancg^Tjc 4 abrasion 5 expulsion 

28. abattoir 1 usurpation 2 cessation 3 legal desertion 4 slaughterhouse 5 nuisance 

29. meticuloms 1 jiartial 2 spacious 3 finical 4 melodic 5 tiny 

30. largess I 1 enormity 2 present 3 monstrosity 4 amiability 5 size 

f Directions and Comi’rehension of Sentences 

AND Paragraphs 

^Only six are included instead of the usual ten, so each 
one • is counted as 1§. 


Read this paragraph. Then read the questions. 
a ( V) check before the best ansiver to each question 
the paragraph again as much as is necessary. 


Make 

Read 


War ship and merchant ship alike clunj? to the coast — 
or if they ventured out to sea, they did so for a voyage to 
be t '‘ounted by the hour, as, for example, from the southwest 
of i Sicily to the opposite coast of Africa — or they relied on 
reg ular trade winds, like the seamen •who sailed from the 
Red Sea to the coast of Malabar going and coming with the 
mor isoons. In spite of exceptions, more apparent perhaps 
thar,i real, such as the voyages of Irish anchorites to Ice- 
land and of the Norsemen to that island, and to Greenland, 
scan lanship continued to be the art of the coaster till the 
clos( ^ of the middle ages. Chaucer’s sailor has hardly lost 
sighi®. of the coast. Such treatises as were written for sea- 
8 
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men were books of pilotag'e. Kxamplos will be found at the 
end of the Hakluyt Society ^s edition of Hues Tractatus de 
glohis. The war-ships, Phoenician, Greek, Roman, Norse, 
Byzantine and Italian, throug^hout the middle ages, used 
sails only when not in action. They were rowed in battle, 
and the mast was lowered, or left on shore. Whenever 
they could they avoided passing the night at sea. Their 
galleys were beached or anchored close to the shore and 
the men landed. We know from Thucydides’ narrat ve of 
the expedition to Syracuse that the crews were landed even 
for their meals; from the chronicle of Ramon de Muntaner 
we know that this was also the case with the best Mediter- 
ranean squadrons at the end of the 13th century’. ^The 
Athenians, clinging to the coast, spent two months in going 
from Athens to Sy’racuse. Roger di Lauria, the admiral of 
Aragon, when coining from Sicily^ in circumstances of great 
urgency to Catalonia, went round by the coast of Africa 
and Spain. When under sails the ships of vrar and of com- 
merce alike had, at the outside, very few sails, and ge tier- 
ally only’^ one great course, square and slung by the middle 
of the yard. It could be ti'ained fore and aft by bowli les, 
so as to enable the vessel to sail on the wind. Under th ese 
restrictions seamanshi]) ivas necessarily a limited urt. 
Prom Marco Polo we learn that tlie seamen of the China 
Sea and of the Indian Ocean ivere coasters like their En ro- 
pean contemporaries. 

31. Put a check before two of the following statemc nts 
which make it almost certain that the Spanish sailors of the 
middle ages were afraid to venture far from land, CJp^eck 
only two. t 

Crew^s landed even for their meals. * 

They did not stay on the sea at night. 

They were afraid of lack of wind. 

Lauria went from Sicily to Catalonia by the coa st of 
Africa and Spain. 

Ships had few sails. 
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Marco Polo’s statement. 

Treatises on seamanship were chiefly about pilotage. 

32. Put a chech before the two of the following which 
best show the fear of the open sea. Chech only two. 

Books were on pilotage. 

Chaucer’s sailor. 

Hues Tractatus de globis. 

In sailing away from land they relied on regular 
trade winds. 

Scant supply of food. 

The sailors landed at night. 

The use of primitive sails. 

33. Tlow was the single sail trained so that the ship 
would go in the same direction as the windf 

At right angles to the long axis of the ship. 

By bowlines. 

Parallel to the long axis of the ship. 

34. How wo"^ the single sail trained so that the ship 
would go at right angles to the windf 

At right angles to the long axis of the ship. 

By bowlines. 

Parallel to the long axis of the ship. 

35. What fact stated in the paragraph gives a measure 
of how near the ships of the middle ages kept to the shore? 

They sailed by the hour. 

Chaucer’s sailor hardly lost sight of the coast. 
Masts were left on shore during a battle. 

They returned at night. 

The Athenians’ trip to Syracuse. 

The crews landed for meals. 

Read this paragraph. Then read the question. Make 
a (V) check before the best answer to the question. Read 

the paragraph again as much as is necessary. 

1 

The church cantata, solo or chorale is indistinguishable 
£y^'m a small oratorio or portion of an oratorio. In Bach’s 
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case many of the larger cantatas are actually called ora- 
torios. 

Many of Baches greatest cantatas begin with an elabor- 
ate chorus followed by a couple of arias and recitatives, 
and end with a plain chorale. This has often been com- 
mented ui)on as an example of Bach’s indifference to 
artistic climax in the work as a whole. But no one will 
maintain this who realizes the place which the church can- 
tata occupied in the Lutheran church service. The text was 
carefully based upon the gospel or lessons for the day; 
unless the cantata was short the sermon probably took place 
after the first chorus or one of the arias, and the congrega- 
tion joined in the final chorale. Thus the unity of the ser- 
vice was the unity of the music ; and, in the eases where all 
the movements of the cantata were founded on one and the 
same chorale-tune, this unity has never been equalled, ex- 
cept by those 16th-century masses and motets which are 
founded upon the Gregorian tunes of the festival for which 
they are written. 

36. What feature is stated as giving some of Baches 
cantatas extraordinary unity? 

They begin with an elaborate chorus followed by a 
couple of arias, and end with a plain chorale. 

If the cantata was short, the sermon took place after 
the first chorus. 

His founding all the movements of the cantata on 
the same chorale tune. 

The text was based on the gosj)el of tlie day. 

The congregation joined in the final chorus. 

The unity of the service was the unity of the music. 

SUB-SERIES Q 
Completions 

1 

1. It must seem to the wisest 

men, when brought into contact with the great thi' 
of nature that they 
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nothing to the infini- 
tude of they are ignorant. 

2. It is a maxim that man 

ever written out of reputation by 

himself. 

3. The American press 

above the moral level of the average 

good citizen, — in no country 

either expect or find it 

so, — ^but it is 

of the machine politi- 
cians in the cities. 

4. David Ilnme founded the 

literary school of English historical writing, and 

of the more important 

doctrines of modern political economy, but also 
a paramount influence on the philo- 

eighteenth 

5. Queen Anne was much to horseracing, 

and not only royal plates to be 

for, ran 

for them 

6. The mere practical man regards favorably 


the results of science, 

the through which these results are 

quite superfluous. 

7. happens 


relations of the Senate and the President are seldom 

cordial, confidential, 

he and the majority r 

the Senate belong to the same party, 

the Senate and the President are rival powersv^hat 
ous .'owthf 

8. Francis Bacon in his will, ‘ 

name and memory, I 
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charitable speeches, 

foreign nations, the 

next ages.” 

9. wonderfully little genu- 
ine inventiveness in the , and perhaps 

of all has been shown in 

of political institutions. 

10. The florid of the debating club or 

pomp of the funeral is 

frequently used by orators when but 

of exposition is desirable. 

Mathematics 

There being only five single mathematical tasks in Sub- 
series Q, each is counted as 2. 

11. Let pp^ mean any flat surface enclosed by straight 
lines, the n denoting the number of sides it has. Let E 
mean equiangular. What is the common name for 
Epp*% 

12. Express in brief form, using I, B and D : “The illumi- 
nation varies directly as the brightness of the light 
and inversely as the square of the distance.” Use 
“ — /l times ’ ’ for ‘ ‘ varies as. ’ ’ 

13. Let n ~ any number 

“ fir—l divided by n 

“ «R = 10 “ “ n 

“ w® = the number raised to the same power as 
itself. 

What does^^^ equal! 

14. L^ ^ 2 , etc., be any numbers. 

Lei^ w be their number, that is, n tells how many m*8 
thT^I® are. 
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Let 8 { ) mean “the snm of”. 

Sim’s) 

What name will you give to — 


15. Let A. D. — the average of the deviations of a set of 
numbers from their average, disregarding the signs 
of the deviations. 

Find the A, D. of 6 ', !), 10, 1 1, 14. 


Vocabulary 

Directions and samples as in Sub-Series N. 


21, radial 

22, sequesfntfe 

23, tartiJily 
2i. apoftee 
2.1. nugatory 

26. sedulous 

27. umbel 

28. assoiorution 

29. abjure 

30. auricular 


1 light 2 agitat(jr 3 straight line 4 root 5 ray 
1 follow 2 petition 3 horseman 4 coniiscate 5 redwood 
1 taiigiliility 2 grace 3 8u])tlefy 1 extensibility 5 manageableness 
1 orbit 2 nadir 3 ellipse 4 culmination 5 zodiac 
3 candy 2 belittling 3 inoperative 4 lump of gold f) hades 
1 muddicil 2 sluggish 8 stupid 4 assiduous 5 corrupting 
1 cluster 2 canopy 3 shadow 4 pigment o ribbing 
3 pluek 2 oath 3 coiitinuuuce 4 purtitiou 5 clcveiiicss 
1 swear 2 recant 3 refuse 4 degraded 5 illegal 
1 golden 2 heard 3 jointed 4 distinct 5 clear 


Directions and Comprehension of Sentences 
AKD Paragraphs. 

The paragraph for questions 31, 32, 33 and 34 is “The 
American State” used in Sub-Series O. There being only 
six tasks instead of the usual ten, each is comited as 1|. 

31. Two words are used several times to indicate com- 
parison. Wliicb are they? 


32. Name three states which are like plants which have 
grown from seeds spread by the wind 


33. What states may properly be thought of as being what 
they are as a result of ordinary political growth? 
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34. What part did nature play in deciding that Montana 
should comprise certain territory? 


Read this and then write the answers. Read it again if you 

need to. 

DIRGE IN WOODS 
A wind sways the pines. 

And below 

Not a breath of wUd air; 

Still as the mosses that glow 
On the flooring and over the lines 
Of the roots here and there. 

The pine tree drops its dead ; 

They are quiet, as under the sea. 

Overhead, overhead 
Rushes life in a race, 

As the clouds the clouds chase ; 

And we go, 

And we drop like the fruits of the tree, 

Even we, 

Even so. 

35. What is as still as the mosses? 

36. Three words in the poem indicate comparison. What 

are they? 

Some of the sub-series intermediate between D and N 
may be found in Chapter VI. 

We could have improved this series at the beginning if 
our resources for work had been more extensive. We could 
improve it still more now with the knowledge which we have 
already gained from using it; and the reader should con- 
sider it more as an illustration of the method than as an 
ideal series of intellectual tasks. It is, however, a reason- 
ably satisfactory series for its purpose, as will be seen. 
One possible criticism we may mention, as it concerns an 
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important general’ question of method which we have not 
yet discussed. 

We have defined intellect as that which produces intel- 
lectual products, succeeds with intellectual tasks. We thus 
include not only the native, inherent capacity which a per- 
son has for such successes, but also whatever education has 
added thereto, and whatever increment of success with in- 
tellectual tasks he has by virtue of working with better in- 
tellectual tools. For example, if A can succeed with the 
tasks of the first four sub-series shown whereas B can suc- 
ceed with only the first three, we credit A with a higher 
altitude or level of intellect than B, even though we may 
be confident that if B had had the advantages of A, he 
would have surpassed him. We are measuring available 
power of intellectual achievement without any specification 
as to its genesis. A person who has acquired the intellec- 
tual tool, reading, probably has a considerable advantage 
over one of equal original capacity who has not acquired 
that tool, in the harder completions and directions. One 
who has studied arithmetic surely has a notable advantage 
in many of the arithmetical problems of our series over one 
of equal original capacity who has not studied it. This 
procedure would bo open to criticism if we should assume 
that the score made in the series is a measure of original 
capacity to grow into or acquire intellect, without proving 
that it did so. We shall not; nothing about the causation 
of the ability measured by the series will be taken for 
granted. 

The procedure will be criticized by others as a failure 
to separate original capacity from the circumstances of 
training and to select tasks which would measure the former 
alone. This is an attractive enterprise, but not, in our 
judgment, so important as the measurement of intellect as 
it actually exists and works. We also doubt whether it can 
be achieved until the latter has been. There is also danger 
that, if we include in a series of intellectual tasks only those 
in whose accomplishment differences of education can make 
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little or no difference, we shall have a collection of freakish 
puzzles, irrelevant to the actual operations of intellect by 
persons twelve years or older in the United States to-day — 
or possibly have nothing at all. 

Whatever be its defects, our series defines intellectual 
tasks and provides us with a rank order of sub-series, each 
of which represents all of Intellect CAVD and nothing but 
Intellect CAVD nearly enough so that its intellectual diffi- 
culty can be measured by the methods of the previous chap- 
ter. The number right out of IOC + lOA + lOV lOD 
tasks at any one level correlates almost as closely with the 
number right out of twenty times as many tasks represent- 
ing twenty levels from very low to very high as its own 
self-correlation permits. Our proof of this statement is 
given in Appendices IV and V. It has to be somewhat ir- 
regular and roundabout, since we have been unable to ob- 
tain records from any individuals attempting the entire 
series. But it is conclusive. 

We have now to consider what theoretical or practical 
significance this Intellect CAVD has. 

THE RELATION OF INTELLECT CAVD TO THE ABILITIES MEASURED 
BY ORDINARY INTELLIGENCE EXAMINATIONS 

The ability which it measures is very much the same as 
that which is measured by the Stanford Binct, or by the 
Otis Self-Administering Group Test, or by the Terman 
Group Test for Grades 7 to 12, or by a combination of these 
three, or by the Thorndike Intelligence Kxamination for 
High School Graduates, or by the I. E. E. Tests of Selective 
and Belational Thinking, Generalization and Organization. 
This last is a selected team of tests representing a general 
consensus of psychological opinion concerning symptoms 
of intellect. That is. Intellect CAVD is very much the same 
as that which is measured by representative examinations 
for so-called general intelligence. The evidence for this is 
the correlations obtained. We report these briefly. 

Using persons sixteen years old or older, with Stanford 
Mental Age of from 28 months to 59 months (all in asy- 
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Ituns for the mentally deficient), we find a correlation of 
.73 (.68 Pearson, .78 Sheppard) between a siimmation score 
for CAVD and Stanford Mental Age, the details being as 
shown in Table 7. This'is in fact probably as close as the 
self-correlation of the Stanford Binet will permit. (In the 
62 cases for which we have a second trial of the Stanford 
Binet, the self-correlation is .53.) If the range were ex- 
tended to include all persons sixteen years old or older, 
this correlation of .73 would rise to about .98.® 

At the other extreme we measured twenty adults, all 
high-school graduates, chosen from professional and cler- 
ical workers, with the Thorndike Intelligence Examination 
for High School Graduates (average of two forms), and 
with an incomplete sampling of Intellect CAVD. The cor- 
relation is about .95, the facts being as shown in Table 8. 
The self-correlation of the Thorndike Examination score 
for this group would bo only about .97^, the correlation 
of one form with the other being .95. So Intellect CAVD 
is nearly identical with the ability measured by the Thorn- 
dike Examination. 

Clark [^25], using 180 pupils in Grades 7 to 12 of the 
Lincoln School of New York City, finds the intercorrela- 
tions stated below among (1) a score based on a selection 
of tasks from the arithmetical and sentence completion and 
information sections of Intellect CAVDI, (2) the Otis Self- 
Administering Group Test, (3) the Terman Group Test, 
and (4) the Stanford Binet. Mental Age. Raw 

Correlation 


Part of Intellect CAVDI with Otis S. A. Test .87 

“ ** Terman Group Test .94 

** “ “ “ Stanford M. A. .78 

Otis S. A. Test with Terman Group Test .88 

“ “ “ ** Stanford M. A. .77 

Terman Group Test with Stanford M. A. .77 


0 The tnean equate variation of the random sample of the Army in Stan- 
ford Mental Age was over 34 months (Memoirs, p. 392) ; that of onr group of 
178 eases was tinder S months. 



TABLE 7, 

The Correlation Between CAVD Sumkation Score and Staneord-Binet Mental Age — in the Case or 178 Imbeciles 
Sixteen Tears Old or Older, of Mental Age 28 Months to 59 Months. 
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The selection of tasks from the three fifths of Intellect 
CAVDI was meagre, so that even these three fifths were 
far from perfectly measured. Nor does one trial with an 
Otis or Terman or Stanford examination measure per- 
fectly what would he found if a dozen alternative examina- 
tions of the same type were used. The correlations of In- 
tellect CAVD or CAVDI with the ability as measured by a 
dozen alternative Otis Tests or Terman Tests would ap- 
proximate to perfect correlation for the group in question. 
The obtained correlations are higher than the obtained 
intercorrelations of Otis, Terman and Stanford, and about 
as high as their self-correlations.‘° 

One hundred and forty-six pupils at the very beginning 
of Grade 6 in one school wore tested with a fairly extensive 
selection from CAVD, each being allowed time enough to 
do all that he could. A summation score was given with 
approximately equal weight to C, A, V, and D. These same 
pupils had been tested some months earlier with the Na- 
tional A and B. The correlation between the summation 
score in CAVD and the score in the National was about .76 
(.81 by Pearson *s, and .71 by Sheppard’s formula). This 
is about as close as the correlation between the National 
and a repetition of itself. If we assume that the variabil- 
ity for a group of constant chronological age 12 is two times 
the variability of this selected grade population, the corre- 
lation for the former would be .92. 

Dr. M. A. May measured a group of about 650 pupils in 
Grades 5 to 8 with a composite of our Vocabulary, Arith- 
metic, Completions, Information and Beading tasks. Pint- 
ner had measured the same pupils with the National Intel- 
ligence Examination. The correlation between a rough 
summation score for the former and the score in the latter 
was .84. This again is about as high as the self-correlations 
of the two would permit. 

Sixty-one college sophomores were measured with Army 
Alpha and with 70 CAVD tasks. The correlation was .71 

Just what these self^correlations would be for the group in question is not 
known; but thej' would certainly' not be on the average much above .QO. 
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by the Pearson and .81 by the Sheppard formula. The self- 
correlation of so short a selection from CAVD will not be 
above .80 within this group, and the self-correlation of 
Alpha will be little, if any, above .80. So the correlation of 
.71 or .76 is about as high as the self-correlations permit. 

Three hundred and eighty-eight pupils in Grade 6 were 
tested with a composite of stock intelligence tests (the I. 
E. R. Tests of Selective and Relational Thinking, General- 
ization and Organization) and with a sampling of about 40 
of the completions and about 40 of the arithmetical prob- 
lems of CAVD, the score for these latter being a summation 
of credits. The correlation was .81. The self-cor relation 
for the I. E. R. tests in such a group will be not over .85 
it will probably be about .80 for the sampling of C and A. 
So the composite of C and A correlates nearly as closely 
with the stock test as the reliabilities permit. 

Intellect CAVD or CAVDI is then no more limited or 
unreal or remote from the practical management of intellect 
than the “intellects’* which are measured by the scores in 
examinations representative of the best present practice. 
It is so nearly the same thing as they that vrhat we learn 
about it will have an application nearly .or quite as broad as 
present practice is. 

THE HOMOGENEITY OF DIFFICULTY CAVD 

We have cured the main defect in the methods of mea- 
suring the intellectual difficulty of tasks which was brought 
forward in the previous chapter. We can make sure that a 
task (always now a composite of many single tasks) mea- 
sures all of Intellect CAVD and nothing but Intellect CAVD 
by correlating it therewith, and then measure its difficulty. 

We have now to consider or reconsider a number of 
other questions. The first is whether the tasks whose dif- 
ferences in difficulty we thus measure do really differ in the 
possession of varying amounts of some one thing which are 

It is .82 for a group of 1,039 boys in Grades 9, 10, and 11, and -86 for 
a group of 16-7ear<old boys in Grades 9, 10, and 11. 
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properly represented by cardinal numbers. Do the sub- 
series or levels (A, B, C, . . . Q) form a progressive, homo- 
geneous series enough like a series of lengths or weights or 
temperatures to be subject to the laws of mathematics? 

We show elsewhere-*® that a sub-series of CAVD tasks 
at any level of difficulty measures closely the same ability 
as the total series (provided the sub-series contains enough 
single tasks fairly to sample the tasks at that level). In that 
sense Intellect CAVD is nearly or quite the same from its 
lowest to its highest levels. It is obviously the same in the 
sense that the tasks are throughout to supply words to com- 
plete sentences, to solve arithmetical problems, to under- 
stand single words, and to understand connected discourse. 

It is also the same in the sense that any person who is 
accustomed to think scientifically about intellectual diffi- 
culty progresses up this series without any sense of shock 
or shift or qualitative change. The progress is, of course, 
not so obviously an increase in the amount of one character- 
istic which does not change its nature, as when one looks at 
straight lines of increasing length or cubes of increasing 
volume or lifts graded weights. But it seems logically 
fairly comparable to one’s experience who looks at very 
irregular and differently shaped solids, such as cups, shoes, 
babies, wheels, steam radiators, chairs, and motor cycles, 
which form a series of increasing volumes. 

Finally, there is a very close correlation between level 
or altitude of Intellect CAVD and range or width of intel- 
lect — between the degree of difficulty at which a person can 
succeed with CAVD tasks and the number of CAVD tasks 
that he can succeed with at any specified degree of diffi- 
culty.’® Bange at a given level is entirely measured by 
number, is a variable varying in nothing whatsoever save 
amount. Whatever correlates so closely with it may be ex- 
pected also to exist as varying amounts of some one quality 
or characteristic. 

Appendix IV and Appendix V. 

13 The measurements of this are reported in Chapter XTTT 
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These facts probably warrant us in using the one term 
Difficulty CAVD to designate the variable, with cardinal 
numbers to designate the varying amounts of degrees or in- 
tensities of it. 

Difficulty CAVD is not the same throughout its varia- 
tions in the sense that any one unit of it can replace or be 
interchangeable with any other unit of it, in the way in 
which one inch or one cent can replace any other. We can- 
not put three tasks, each of difficulty 4, together and have a 
task of difficulty 12, nor can we subtract some part of any 
task of difficulty 10 from it so as to leave a task of difficulty 
9 and add that fraction of the actual task to a task of diffi- 
culty 5 so as to make its difficulty 6. The meaning of arith- 
metical operations upon numbers representing degrees of 
difficulty of intellectual tasks must be considered with refer- 
ence to the realities which these numbers represent. 

This is the case, also, •with numbers representing many 
variables such as volts, degrees of temperature, wave- 
lengths or ages, to which arithmetic is none the less usefully 
applied. Dividing a temperature of 300 degrees (above the 
absolute zero) by 10 gives 30 degrees in a certain real and 
useful sense, but not in the same sense that dividing 300 dol- 
lars by 10 gives 30 dollars. We cannot take 5 years of age 
from 40 years old and use it to make a five-year-old into a 
ten-year-old. Multiplying short vibrations will not give 
long "vibrations. 

We shall return to a consideration of the applicability 
of arithmetic to the numbers representing different degrees 
of difficulty CAVD later, after we have gained more knowl- 
edge of them and shown more facts concerning the realities 
for which they stand. For the present it may serve to note 
that the numbers representing difficulty CAVD are roughly 
comparable to the numbers representing temperatures in 
respect of the meaning and use of arithmetical operations 
performed upon them. 

9 




rnTT*. . ;^TIOiT OP A GBADP 

TSl*. ^T-pTFERENCB FROM THE FORM OF DISTRIBl)^ 

popi. dExamistation 

IN STANDARD INTELLIGENCE , 

cp. IlCJTION IN 

®^’^RES TO THE FORM OF ITS DISTRIB ' 

Tj. II’ CAVD 

^ VEL OB ALTITUDE OF INTELLECl 

The next needs considerati concerns our 

use of the scores gtock intelligence 

investigations of tl,^ of distribution oif ''“f ^ons 

of an individual; ana of - ^ae individuals 

in Grade 6, Grade 9, and Grade ^2* and of tf^® relation of 
the variahilitj’ of an individual to avers degree of 
intellect. The scores in any one of the.g^ stock * 
represent the composite influence of le\e,| ^dtitude, range 
or width, and facility or speed, in unknown ^^pportions. In 
strict logic we should have used for our pu:i^*Pose a large 
number of examinations, each concerned with^ level o c alti- 
tude alone, but made by different experts and lifvithoii p<^rej- 
udice concerning the form of distribution of ^mtelect,"- as 
was the case with the material which we did usci Tie fo‘^>rm 
of distribution which we really needed for our ai-gmnent is 
the form of distribution in respect of how hard things IT-he 
varying intellect of an individual and the varying intellei^.ts 
of a group can succeed with. It would, however, have bee5n 
utterly impracticable to have attempted to have expert jS 
make seventeen such examinations, and to have applied 
these to the large number of individuals needed to make the 
argument valid. The cost in time and labor would have been 
prohibitive. We therefore used the stock examinations 
which were available; and set up experiments to ascertain 
how closely the ability measured by these examinations is 
correlated with altitude or level of intellect as measured by 
the hardest intellectual tasks at which a person attains a 
given percentage of successes. Are we justified in infer- 
ring the form of distribution of level or altitude of Intel- 
lect CAVD from the form of distribution of the ability 
measured by these stock intelligence examinations? 

It is impossible to answer this question by a straight 
forward experiment in which a group of several hundred 
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pupils of, for example, Grade 9 should be given the graded 
tasks of Intellect CAVD and also be tested with a dozen or 
more stock tests of intelligence. The form of distribution 
found for the group in level or altitude measured by the 
graded tasks of Intellect CAVD depends upon the real dif- 
ferences in difficulty between the tasks. This is precisely 
what we are trying to determine. We must argue indi- 
rectly, as by the correlation between level or altitude of 
Intellect CAVD and score in a stock test of intelligence. 

We have already shown that, in general, summation 
scores in Intellect CAVD measure very closely the same 
ability as the stock intelligence examinations do. Conse- 
quently we may correlate a level or altitude score in Intel- 
lect CAVD with a summation score in Intellect CAVD. 
Such correlations will have closely the same meaning as 
correlations veen a level score in Intellect CAVD and 
the official score in the stock intelligence examination. If 
the correlations are close, the abilities are similar and will 
have, in any given group, similar forms of distribution 
when measured in truly equal units. 

The correlations between altitude or level score in 
CAVD and summation score in CAVD are reported in 
Chapter 13. They are well above .90 even in groups of 
rather narrow range. The correlations between altitude 
or level score in CAVD and official score in standard intelli- 
gence examinations are reported here. They show in gen- 
eral that if an individual is tested with any sufficiently ex- 
tensive collection of intellectual tasks representing wide 
variations in difficulty, the level w^hich he reaches is closely 
correlated with the score which he obtains by a summation 
of credits for work done within a certain time, after the 
fashion common in the stock examination. 

In the case of 146 pupils of Grade 5^, the correlation 
between altitude or level score in CAVD and score in the 
National Kxamination (A + B) was .72 by one determina- 
tion of altitude and .665 by an independent determination 
of altitude. The two independent measures of altitude or 
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level correlated .755. The level score in CAVD thus corre- 
lates with the National Intelligence score nearly as closely 
as with itself (.69^ compared with .75^). 

Twenty individuals were measured in respect of a sam- 
pling from Intellect CAVD by the level at which they could 
succeed with fifty percent of the task elements, that is, could 
do correctly 20 of the 40 single tasks making up a composite 
task. Each was measured also by two alternative forms of 
the Thorndike Examination for High School Graduates. 
The correlations between the level score and the two stock 
examination scores are .92 and .90. One of the stock ex- 
amination scores correlates with the other .95 ; and the level 
score would probably not correlate with that from another 
similar sampling from Intellect CAVD more than .97. So 
perfectly measured level or altitude CAVD and perfectly 
measured ability in this sort of stock examination would 
probably correlate about .95. 

In this same group the correlation between the level 
score in CAVD and a summation score in a composite of 
C, A, V, and D, including the tasks used in determining the 
level score and many others, was .96. The correlations 
between this summation score and the scores in the two 
Thorndike Examinations were both .95. Thus level score 
in CAVD correlates with summation score in CAVD nearly 
or quite up to its probable self -correlation, and correlates 
with summation score in the Thorndike 91/95 as high as 
does the summation score from much more extensive 
testing. 

Dr. J ohn R. Clark secured measurements of one hundred 
eighty pupils in Grades 7 to 12 of the Lincoln School in 
the Stanford Binet, Terman Group Test and Otis Self-Ad- 
ministering Group Test, and in rough measures of level or 
altitude, range or width, and speed or facility in samplings 
from the arithmetic and completion tasks of Intellect 
CAVD. He found the correlations with level to be as 
shown in Table 9. 

The three stock examinations correlate on the average 
.80 one with another, and correlate on the average .76 with 
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arithmetic level score if the latter is perfectly measured, 
and .64 with completion level score if that is perfectly mea- 
sured. They may fairly be expected to correlate almost 
perfectly with a level score for the sort of tasks which they 
themselves contain. 


TABLE 9 

The Correlations between Scores in Stock Intelligence Examinations 
AND Level Scores in Arithmetical Problems and Sentence Comple- 
tions. 180 Pupils in Grades 7 to 12. Bata prom Clark (’24). 



OD 

<4.3 

o 

Terman 

Arith. Level 

Co. Level 

Arith. Level if 
perfectl 7 measured 

Co. Level if 
perfectly measured 

Stanford Binet 

.74 

.77 

.57 

.57 

.65 

.65 

Otis Self Adm. 


.88 

.74 

.55 

.83 

.61 

Terman Group 



.74 

.59 

.80 

.66 

Arith. Level 




.46 




Arith. Level perfectlj 

measured .55 


We may then assume with risk of only moderate error 
that achievement with the series of CAVD composite tasks 
represents an important ability closely allied to that which 
such stock intelligence examinations as the Stanford Binet, 
National, Otis, Terman, and Thorndike measure; that the 
level or altitude score attained in Intellect CAVB may 
properly be expressed as a cardinal number ; that this level 
score, if measured in truly equal units, will show a rather 
close approach to Form A (the probability surface) in 
Grade 6, Grade 9, Grade 12, or any intervening grade ; and 
that the correlation between a personas true level or alti- 
tude in Intellect CAVD and his score in any one of the sub- 
series or composite tasks (A, B, C, . . . Q) will be very 
nearly as close as the self-correlation of the sub-series will 
permit. 
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We can then measnre the differences in difficulty be- 
tween any two of these composite tasks for any grade group 
between 6 and 12 ■which is so constituted that all of its indi- 
viduals do not succeed, neither do all fail, with either of 
the two composite tasks in question. 



CHAPTER IV 


The Measurement of the iNTErj.ECTUAL Difficulty of a 

Single Brief Task 

In the previous chapter we have shown that if intel- 
lectual difficulty is defined as the difficulty of a perfectly 
representative suh-series of a total series of concretely spe- 
cified tasks, it can be measured, at least over a range from 
tasks which the stupidest children in Grade 6 can do to 
tasks which only the brightest of college freshmen can do. 

In the present chapter we have to consider the difficul- 
ties which confront us when the sub-series is only imper- 
fectly representative of the total. As the title of the chap- 
ter indicates, we shall emphasize the extreme cases where 
the task is a very partial representative ; but the discussion 
will provide also for the treatment of any degree of par- 
tiality and incompleteness. 

We have seen that when we know that k percent of a 
group succeed with a task ti (k being > 0 and < 100) we 
may express the difficulty of the task as Mti + Aoti where 
Mt is the central tendency of the group in the ability mea- 
sured by t, cti is the variability of the group in the ability 
measured by t, and A is a factor dependent for its sign and 
absolute value on k. We have seen that we cannot, wdthout 
further knowledge to that effect, assume that Mti is equal 
to the central tendency of the group in intellect or anything 
else save the ability measured by ti ; or that ati is equal to 
the variability of the group in general intellect or anything 
else save the ability measured by t,. 

the problem in the case of single tasks, each of which 

MEASURES INTELLECT PLUS A MERE SAMPLING ERROR 

We have now to consider the possibility of such further 
knowledge. Consider it first for cases where tj is a repre- 
sentative sample of intellectual tasks, and the measurement 
afforded by ti is a compound of perfectly measured intel- 
lect and error of sampling, and the error is of the same 
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magnitude for any one ti as for any other, so that the aver- 
age of a sialBcient number of ti’s would be a perfect measure 
of intellect, so that the correlation of any one ti with any 
other will be a constant. For such cases the knowledge 
needed is available and Mi and Oi, the central tendency and 
variability of the group in intellect, can be computed from 
Mti and oti, when the amount of the error is known. 

Since Mi is the average of the N individuals of the 
group, each measured in (ti -h tg + tg -f - 14 • • • t„)/n, M, 
will, if N is large, approximate closely to Mu, Mta, Mts, etc., 
and any one of these will approximate closely to any other 
of them. The effect of the error whereby the estimate of 
intellect by any t differs from that by the average of all 
the t’s is as often plus as minus, and is negligible for our 
purposes so far as concerns the central tendency of a large 
group. M, may be taken as equal to Mti. 

Because of the sampling error, ot will always be smaller 
than 0 ti. Oti will equal Vup“ Oe*? where Op is the variabil- 
ity of the N individuals each measured by (ti + t 2 tg 4" t 4 
• * • t„)/n, and Oe is the variability of the sampling error, 
dependent upon the variations of ti, tg, tg, etc., in any indi- 
vidual from the average of t^, t 2 , tg, etc., for that individual. 
CTe may be computed in various ways from various measures 
of the unlikeness of tj, tj, tg, etc., in the same individual, 
such as the correlation of ti with t, in the group, or the cor- 
relation of ti or ta or tg with (ti -}- tg + tg ■ * • 1 ^)/n, or the 
differences between ti and ta in individuals, or the variabil- 
ity of an individual in intellect as estimated first by ti, then 
by ta, then by tg, and so on. 

Thus 01 = GtV Tta tp or 01 = — £L= [Kelley, ’23, formula 

y^ta tb 

166, p. 213] and since Th = Vrt*tb [Kelley, ’23, formula 160, 
p. 206], Gt where 

1 This second f ormnla is presented because it lends itself better to much of 
the material at our disposaL It is derived bj Kelley directly from Spearman’s 
formulas fox the correlations of sums or averages. The reader of less mathe- 
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Q, = the variability of the group in intellect, which is 
here identical with Op, the variability of the group in (tiH- 
t 2 + t8 • • * t„)/n. 

rti = the correlation between the estimate of intellect by 
any one t in question and the estimate of intellect by (ti + 
t 2 + ts • • • to)/n in the group in question. 

at = the variability of the group in the ability measured 
by the one t in question. 

rtatb = the correlation between the estimate of intellect 
by any one t and that by any other t. 

For example, assume that the completion task 22 is one 
taken at random from a number of completions, each of 
which measures intellect plus a similar sampling error, the 
average of all of them measuring it exactly. 

We found the difficulty of task 22 to be .098 times the 
022 of the ninth grade group harder than the M 22 of that 
group. In accord with our assumptions, we may replace 
M 22 by M,. The correlation between score in task 22 and 
intellect may be taken as approximately .40 for the group 
in question, since the obtained correlation with a fairly 
close representation of intellect is .37^. In place of .098 022 

we then put .098-^. The purely intellectual difficulty 

of task 22, freed from the effect of the sampling error, is 
now measured as .245 (oi tor 9 tn grade) and can be compared 
with that of any other task representing intellect plus the 
effect of sampling error for which we have the percent of 
correct responses in this group. Thus task 20, which 

znatical ability may easily derive it from tlie more familiar formula for the 
correction for attenuation^ as follows: 

Consider the ordinary Spearman attenuation formula for our case, 

IjOt and be perfect measures of intellect. Then ri^i^=:1.00. Tt.oi is 1.00 
bj hypothesis. 

Bo V^,7^ = Tti. 

Bubstituting in o, = — —■ — we have 0 , = — = — - in this ease. 

r„ 
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showed 60% of correct resjwnses in this gronp, giving 
— .2533020) has a correlation with the same fairly close rep- 
resentation of intellect just mentioned, of .18^. Its corre- 
lation with intellect may be taken as approximately .20. 
Assuming that it is a random sample from a set of tasks 
whose average measures intellect perfectly,® and each of 
which suffers an error of equal magnitude, we transmute 

.2533 Oio 

'‘.2533 O 20 easier than Moo m Grade 9^' into 2 q — 

easier than M, in Grade 9/’ or — 1.27 oio. Task 20 is then 
1.51 easier than task 22, the unit of measure being the mean 
square deviation of intellect in Grade 9. 

If the error whereby the ability measured by a task dif- 
fers from intellect is a random sampling error, so that per- 
fectly measured intellect can be got by merely increasing 
the number of tasks strictly comparable to it drawn in the 
sample, we can then correct for it, the correction being a 
further application of the facts shown by Spearman [’04, 
’07, ’10, and ’131, Boas [’06], Thorndike *^[’13], and Kelley 
[’19, ’21 and ’23]. 

If the single tasks whose intellectual difficulty we wish 
to determine measured intellect perfectly, except for such a 
random sampling error, we could and should compute 
r,, (or rtifcAVD)) for each of them in each group used, and 
apply the corrections. 

The effect of the correction may be illustrated by cases 
where we have reduced the sampling error empirically by 
using ten tasks in place of one. 

Thus for 250 pupils in Grade 8^, the median of the ten 
percents correct for the ten single word tasks and the 
percent scoring five or more correct responses out of the 
ten was as shown in Table 10 for each of the fourteen 10- 
word composites in the I. E. R. A-2 and B-2. Table 10 

2 This average will have to be computed from a larger number of t's than 
would be needed in the case of the tasks from which task 22 was drawn as a 
random sample, since the error is here larger, making the correlation with 
i smaller. 
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also reports the oi values and the Oio values which corre- 
spond to these percents. The percent is more remote 
from 50 when we shift from one right out of one to five or 
more right out of ten ; and the value in terms of Oio is more 
remote from the median of the group. 

TABLE 10. 

Thk Etfect op Decbeasing the Eeboe op Estimating the DiPPicuiiTY op the 
Mepian Task op a Composite of Ten by the use op the Percent of a 
Group Scoring ^‘5 or More Bight out op Ten'' in Place of the 
Median op the Ten Percents op the Group Scoring 
^ bright" in the Tasks Taken One at a Tims^ 

Vocabulary Tasks in the Case op 250 
Pupils op Grade 8%. 


Distance from the Median 



Median of 
the ten 
Percents 

Percent 

Scoring 

5 or more 

Ability of 
In Terms 
of 0 ^ 

tbe Group 

In Terms 

Composite 1 

93.2 

98.0 

-1.49 

-2.05 

la 

93,4 

97.6 

-1.51 

-1.98 

2 

82,6 

92.0 

-0.94 

-1.41 

2 a 

87,4 

97.2 

-1.15 

-1-91 

'' 3 

62.8 

74.0 

-0,32 

-0.64 

3a 

72.3 

86.4 

-0.60 

- 1.10 

4 

55.2 

64.4 

-0.13 

-0.37 

4a 

65.6 

61.6 

-0.14 

-0.30 

5 

43.0 

44.4 

+ 0.18 

+ 0.15 

5a 

43.2 

44.8 

+ 0.17 

+ 0.13 

6 

23.4 

15.2 

+ 0.73 

+ 1.03 

6 a 

25.8 

19.6 

+ 0.65 

+ 0.86 

'' 7 

15.8 

4.4 

+ 1.00 

+ 1.71 

7a 

10.8 

.8 

+ 1.24 

+ 2.41 


It may he realized more exactly by applying the formula 
to a few representative cases. Thus, tasks A, B and C, 
each being done correctly by the same percent (80) of a 
group (of normal form of distribution), but correlating 
with intellect to the extent of .20, .35 and .50, respectively, 
in that group, will be of intellectual difficulty — 4.208 oi, 
— 2.405 oi and — 1.683 oi, respectively. Tasks C, D and E, 
although done correctly by very different percents of the 
group, are of equal intellectual difficulty, their differences 
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in difficulty being counterbalanced by reverse differences in 
intellectualness. 


Task 

% suc- 
cessful 

Correlation 
vrith. intellect 

Difficulty 

Intellectual 

Dif8.culty 

A 

80 

.20 

- .8416cTj^^ 

- 4.2080] 

B 

80 

.35 

- .8416ob 

-2.4050^ 

C 

80 

.50 

- .8416ac 

— 1*6830| 

D 

60.1 

.20 

— .25590]} 

- 1.28O0J 

E 

67.3 

.35 

- -448205; 

- 1.28O0J 

P 

73.9 

.50 

- .6403op 

- 1.2800, 


THE PROBLEM IN THE CASE OF SUCH SINGLE TASKS AS ARE USED 
IN CAVD OR IN STANDARD INTELLIGENCE EXAMINATIONS 

Unfortunately we cannot be sure that a single task will 
measure intellect save for such a sampling error. This may 
be best realized by taking our Intellect CAVD as intellect 
for the moment, and considering a task made up of 20 com- 
pletions, 20 arithmetical problems, 20 words and 20 direc- 
tions, all of equal difficulty. The ability measured by such 
an 80-element task, if the elements are well selected, is ap- 
proximately perfectly representative of ability CAVD. 

Now if we take one of the eighty tasks at random, we 
do not have something which measures what the eighty 
together do plus an ordinary error of sampling. One 
word-knowledge test does not differ from one arithmetical 
problem test in the same way that one arithmetical prob- 
lem test differs from another. The total is too varied a 
synthesis and the single task is too small a sample for the 
latter to represent the former plus an ordinary sampling 
error. In the eighty are four different sorts of tasks; in 
the twenty completions there may be four or five which 
require knowledge of specialized facts ; amongst these four 
or five, there may be one which is very much easier for in- 
tellects which have lived in the country than for intellects, 
otherwise similar, which have lived in the city ; and another 
of which the reverse is true. 

The case is not so much like measuring a man’s height 
a dozen times and taking one of the dozen to represent 
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their average, as like measuring his head, his neck, his 
trunk, his legs to the knees, his shins, and his feet and 
adding the results to get his height. Our measures of in- 
tellect are inventories; we combine C, A, V and D as we 
might combine a man’s real estate, ships, stocks, bonds, ac- 
counts receivable, merchandise, materials, and cash on 
hand. What he happened to own in the way of real estate 
in Boston would not in a useful sense represent his total 
wealth plus a sampling error. 

Assume for the purpose of illustration that: (1) intel- 
lect is composed of C and A in equal parts, and is perfectly 
measured at the level in question by a task composed of 20 
completions and 20 arithmetical problems, the two twenties 
having equal weight; ( 2 ) a task comprising the 20 comple- 
tions will correlate perfectly with a task comprising 100 
completions from which the 20 are a random sample; (3) 
a task comprising the 20 arithmetical problems will corre- 
late perfectly with a task comprising 100 problems from 
which the 20 are a random sample; (4) the 20 completions 
or the 100 completions will correlate 0 with the 20 arith- 
metical problems or the 100 arithmetical problems. 

If now N individuals composing a group distributed 
“normally” are measured in respect of their success with 
a task composed of 40 completions, and if a given percent 
succeed with the task (that is, have 20 or more of the 40 
right), the difficulty of that 40-completion task is M 40 C + 
XCT 40 C- The correlation between the score in 40C and the 
score in intellect, or C + A measured by 20C + 20 A, is .707. 
The correlation between the score in 40C and the score in 
another 40C is 1 . 00 . 


By our assumptions 


O 2 OC + 20A "N/ <T*20C “I” 

since 

^*200 20A 0 * 

^200 + 20A “ — 2(7200 

since 

G 2 OC <T20A- 

^ V2 

O 20 C + 20 A — — ^ — cr4oc 

since 

CT40C 2 (J 2 oc> since r 2 oc 20 c 
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But by the formula (at= — ==| we should have 

\ Vrt,t,/ 

G20C + 20A . . 

0400= -7=—^, giving 

V 1^200 20c 


O4OC 


0200 f 20A 

~Vi~ 


or 02OC + 20 A CJ40C* 

We see the reason for the discrepancy if we consider 
the attenuation formula 


1*C<« w 


1*040 with 020 + A20 


tth (O 4 A) « 


V" 1*040 C40 ^(030 + A20> (C20 + A20) 


With our present assumptions, Tc with (c + a)« is not 1.00, 
because, no matter how many C^s we take, we do not get 
all of intellect and nothing but intellect. It is in fact .707. 
So we do not have 


^C40 w Ith C40 ^C40 with C20 + A20> 


but 


^ 1*C40 with 020 + A20 

V 1^040 with 040 JQJ 

If we substitute r^^o with 020 + a2o for '\/rc4oc4(, in the 

formula at== — have again the erroneous result 
Vrt,t, 

O 2 OC + 20A CT40C* 

Now tlie correlation between C_ and (C4-A + V + 
D)* or any other form of perfectly measured intellect is not 
perfect ; and the correlation between C « and either A « or 
V « or D „ or Picture Completions « or Geometrical Rela- 
tions 00 is not in fact perfect. In general, if we sample by 
taking one small task, it has to be so limited that if we take 
a thousand tasks closely like it, the score therein need not 
correlate perfectly with the score in intellect, or with the 
score in a thousand tasks closely like any other one task 
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with which we might begin. In particular, no single com- 
pletion, or word to be defined, or problem in arithmetic, or 
sentence to be comprehended can safely be regarded as dif- 
fering from intellect only by a sampling error such as 
may be adequately corrected for by 

o, Oi 

Ot = —~=^OT Ot — — • 

■\/rt,t, rti 

A single task, ti, measures not a large part of intellect 
plus a small error due to the action of a large number of 
factors of about equal magnitude, but a small part of in- 
tellect plus a large error. The latter is due to the action 
of factors some of which, like residence in the city, access 
to books, formal training with arithmetical problems, spe- 
cial acquaintance with the particular word or sentence or 
problem, may be of very great magnitude in comparison 
with others. 

More generally, a, = Ot Tti or Ot == — Kelley’s dis- 

cussioii [p. 213] makes clear, true for a case where i is 
simply the average of many t’s, each of which has closely 
the same o as any other and closely the same rti as any 
other. It is not true when we fail to get i by a collection 
of tasks however extensive. And no matter how many 
completions we take, we shall never get an i made up of 
completions and arithmetical problems unless the corre- 
lation between sentence completion and solving arithmeti- 
cal problems is perfect. 

The quantitative importance of having a varied as well 
as a large sample may be illustrated by measurements of 
the correlation between i, as represented by the summation 
score in CAVD (40C + 40A + 40V + 40D), and Composites 
of 10 made up all of C or A or V or D on the one hand, and 
on the other, composites of 10 made up of 2C + 3A + 2V 
+ 3D or of 3C + 2A + 3V -|- 2D. In the case of 240 col- 
lege graduates, the average of the former sort was .59 with 
a P.E. of =t .028 ; the average of the latter sort was .72 
with a P.E. of db .022. 
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It should also be noted that, even if the correlation be- 
tween the score in an infinite number of completions and 
the score in an infinite number of arithmetical problems 
were perfect, so that we got all of intellect and nothing but 
intellect as well by a sampling of one type of task as by a 
mixed sampling (the reduction of rti below 1,00 being due 
purely to sampling error), still the practical difficulties in 
the way of applying the correction would make it far wiser 
first to construct composite tasks. It is very laborious to 
compute rti for each element. It will be low (roughly 
from .20 to .60 for a group of individuals in the same school 
grade), and the probable error of a low bi-serial r is such 
that an enormous number of individuals must be tested to 
obtain r^ with a precision such that the probable error is 
less than .01 (from 5,000 to more than 10,000 for r = .40, 
according as the split of successes and failures is near .50, 
.50, or remote therefrom). 

THE SOliXrTION BY THE XJSB OF EXTENSIVE COMPOSITE TASKS 

The only safe and wise course is, then, to make sure that 
the tasks whose difficulty we are to measure are alike in the 
amount of intellect which each involves, and in the amount 
of non-intellect by which each is contaminated, by using 
composite tasks each containing many single tasks, repre- 
senting with proper weight the various aspects or constitu- 
ents of intellect. The nearer we come to having each of 
them measure all of intellect and nothing but intellect, the 
safer our course will be. 

With composites whi^ differ from i only by the sam- 
pling error the correction formulas are appropriate. In 
proportion as the composite is made to include a large 
sampling, the labor of computing rti or rt,t, to a given de- 
gree of precision is reduced and the reliability of the cor- 
rection is increased. With forty-element CAVD com- 
posite s, for example, it is safe to infer Oi from ot,, either by 
Cfi "v/rtitjOt, or by Oi = rtiiut,. 

In constructing composite tasks whose difficulty will be 
truly intellectual difficulty, freed from the sampling error 
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by having many tasks, and freed from the constant error 
by having a proper representation of all the elements of 
intellect, it may be desirable, other things being equal, to 
include in any one composite only tasks which would show 
approximately the same intellectual difficulty if, by a mir- 
acle, all of intellect, and nothing but intellect, could in each 
case be utili 25 ed for success. The measure of the difficulty 
of a composite of n tasks would be more reliable if this 
could be the case. The construction of composites of speci- 
fied amounts of difficulty would be less a matter of trial and 
correction. 

So, for this purpose, we may need to measure approxi- 
mately something which, for lack of a better name, we may 
call the ‘‘intellectual difficulty’^ of single tasks, and to know 
how close the approximations are. 

The facts which we shall present in this connection are 
also of importance in estimating the errors in scales® 
which have been constructed on the assumption that ot^, 
Ot„ ot„ ot^, etc., are equal. They are also of importance in 
connection with the general technique of selecting single 
tasks to make a composite, even if we make no attempt to 
select them to be of the same intellectual difficulty, rather 
than of the same difficulty. 

These facts are the percents of some group succeeding 
with the several tasks (ti, tg, tg, etc.) whence we may com- 
pute M measures of their difficulty (Mt, + C, at,, Mt, + 
Cj at„ Mt, + C, Of,, etc.) ; and the correlations (rt,!, rt,i, 
rtai, etc., between each of many single tasks and intellect 
(GAVD or some other defined intellect), whence we may 
compute the extent to which tj, t2, ta, etc. represent intellect, 
and so estimate their “intellectual difficulty.” We have 
seen that with a large group, Mt„ Mt„ Mt„ etc., will be 
closely equal. In proportion as rt,i, rt,i, rt,i, etc., are ap- 
proximately equal, at,, at,, at,, etc., will be approximately 
equal, and ai will be approximately the same fraction of 
each of them, equalling respectively Ei*, Vui* + E2®, 

s Bach aa the Buckingham Spelling Scale^ Trabue Completion Scalea, Van 
Wagenen Hiatory Beales. 

10 
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Voi® 4- i^V, where Ei or Eg or Eg is the “error*’ by which 
the estimate of intellect by the single task diverges from the 
estimate of intellect from a properly weighted sum of all 
tasks. El, Eg, Es, etc., will be approximately equal, if they 
produce approximately equal reductions from perfection in 
the correlations rt^i, rtji, rt.i, etc. 

If, then, we select single tasks which are done by equal 
percents of a large group, and also are approximately 
equally closely correlated with intellect, we shall have 
equality in the sort of intellectual difficulty which we are 
discussing. For example. Table 11 shows, in the case of 30 
reading tasks, the percents succeeding and the correlations 
(bi-serial r) with the combined score in two forms of a 
standard intelligence examination given a year apart (the 
I. E. R. Tests of Selective and Relational Thinking, Gener- 
alization and Organization*). The facts are given for 668 
pupils in Grade 11. Using the facts of Table 11 as our 
guide, tasks 10, 15, and 24 may be expected to be of approxi- 
mately equal “intellectual difficulty.” They are approxi- 
mately equally difficult because the percents succeeding 
are respectively 66, 65, and 67. They are approximately 
equally intellectual because the rtit’s are, respectiv(*ly, .40, 
.41, and .38. We can also balance low degrees of %s (per- 
cent successful) against high degrees of rti so as to get 
tasks that would be of equal intellectual difficulty in so far 
as the formula is applicable. 

Even if it is not desirable to spend time in choosing 

tasks which are alike in the H values of o, as inferred 

from a I = TtiOt, it will be very useful to know how much 
difference will be shown in the rti’s of single tasks in com- 
pleting sentences, solving arithmetical problems, knowing 
word-meanings, following directions or answering ques- 
tions about a paragraph, giving opposites, possessing and 
using information, completing pictures, supplying or se- 
lecting the proper related term as in the analogies test, and 
other stock forms of tasks used in instruments for measure- 

1.70 

* The self -eorrelatiozL of this combined score is approximately *^^9 

in this group. 
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luent of intellect. For, other things being equal, the higher 
Tti is the more suitable the task is for inclusion in a com- 
posite to measure i. 

TABLE 11 

The Dhticulty and iNTKLLBCTTTAiiNEss or 30 Single Tasks in Understand- 
ing Sentences, Measured by the Percent op 668 11th Grade Pupils 
Succeeding with Each, and by the Correlations op Success 
IN Each with the Average Score in Two Forms op 
THE I.E.B. Sel. Eel. Qen, Oeg. Examination. 


Task 

% Suc- 
ceeding 

Ttl 

Unreliabiljty 

of It, (Or) 

DI A 1 

93 

.43 

=t .07 

2 

94 

.30 

.075 

3 

91 

.29 

'' .065 

4 

84 

.55 

“ .04 

5 

76 

.36 

.045 

6 

83 

,45 

.045 

7 

75 

.43 

.04 

8 

82 

.53 

"" .04 

9 

82 

.54 

.04 

10 

66 

.40 

.04 

11 

81 

.52 

“ .04 

12 

69 

.48 

“ .04 

13 

77 

.45 

.04 

14 

57 

.28 

.045 

15 

65 

.41 

‘‘ .04 

16 

75 

.48 

.04 

17 

66 

.45 

.04 

18 

65 

.46 

.04 

19 

75 

.40 

.045 

20 

70 

.49 

.04 

21 

69 

.36 

.045 

22 

56 

.33 

.04 

23 

54 

.35 

.04 

24 

67 

.38 

.04 

25 

70 

.32 

.045 

26 

66 

.52 

.04 

27 

64 

.46 

.04 

28 

64 

.49 

.04 

29 

57 

.45 

.04 

30 

64 

.53 

.036 


THE CORRELATIONS OF SINGLE TASKS WITH MEASURES OF 

INTELLECT 

We have made the computations in the ease of 24 other 
reading tasks and 55 vocabulary tasks, with the results 
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shown in Table 12. We also give in Table 13 the facts for 
10 completion tasks, 10 arithmetical tasks, and 10 vocabn- 
lary tasks, using a group of 240 college graduates. These 
data, together with those of Vincent (to be described 
shortly), make possible a general estimate of how much r^t 
may be expected to vary in the case of single tasks selected 
or devised by jjsychologists as suitable elements of an in- 
telligence examination. 

The obtained correlations vary very widely, but some 
of this variability is due to the unreliability of the deter- 
minations; and allowance must be made for this in order 
to estimate the true variation in r^ due to the differences 
among single tasks in the amount of i which each involves 
and the amount of non-i by which it is contaminated. 

Consider first the facts from 99 tasks in reading and 
vocabulary, where the percent is between 5 and 95, in the 
case of 668 and 454 pupils, respectively, in Grade 11, shown 
in Table 14. We omit the very, very easy and very, very 
hard tasks, since we should measure their difficulty by a 
duller and by a brighter group, respectively. 

It is obvious to inspection that the correlations vary 
more than can be accounted for by their unreliabilities. In 
the .40 to .60 group, we have a range from — .45 to .52, in 
the .60 to .80 group, a range from .11 to .56, and in the .80 
to .95 group, a range from — .02 to .67.® 

The variation which we should obtain with the unrelia- 
bilities cut to 0 by a sufficiently large group is to be found 

from Otnie == Vo^obt. — o^orror- Usiug mcdiaus as central ten- 
dencies, the facts are: 

Otnie . 40 - 80 = V *<^217-0024 07 .139 (n = 20). 

Otruo . 60-80 = V -0092-0022 or .084 (n = 35). 

Otrue . 80 - 96 = V-0246-.0041 07 .143 (n = 38). 

® The bi-serial r's in Tables 11, !12, and 13 were computed by an approxi- 
mate method. They will diverge from r'e computed accurately by not over 
.006, which is not of consequence in comparison with the variations which we 
are considering. 
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TABLE 12 

pEBCBITTAaB SUCCEBOIKO AND COKREIiAOMONS WITH A CBITBIUOH IK THE CASE OE 

24 Biahinci Tasks and 52 VocABiriiART Tasks: Gbade 11: n = 668 
roK the Beadino Tasks and 454 fob the VocabuiiAbt Tasks. 


Task 

% 8 

mam 

Task 

% 8 

BBi 

DI A 66 

72 

.39 zt .045 

VB 45 

87 

.29 ± .07 

67 

81 

.28 .045 

46 

85 

.39'* .06 

68 

83 

.31 “ .05 

47 

85 

.45"* .06 

69 

88 

.39 ‘‘ .055 

48 

97 

.40"" .11 

70 

80 

.24 .045 

49 

20 

.15" .06 

71 

94 

.30 .075 

50 

66 

.33" .055 

72 

42 

.13 .05 

51 

77 

.24" .055 

73 

80 

.23 .045 

52 

73 

,52" .05 

74 

81 

.41 .045 

53 

26 

-.13" .06 

75 

53 

.39 .04 

54 

65 

,27" .055 

77 

69 

.35 .045 

55 

84 

-.02"" .07 

78 

90 

.31 .065 

56 

96 

.58"" .085 

79 

73 

.32 '' .045 

57 

44 

,48" .05 




58 

80 

.48" .05 

DII A 1 

47 

.17 .045 

59 

90 

.36" .07 

2 

41 

.35 .045 

60 

82 

.50"" .05 

3 

34 

.13 .05 

61 

87 

.54" ,06 

4 

54 

.22 .045 

62 

66 

.29"" .055 

5 

51 

.25 .045 

63 

43 

.35" .05 

17 

22 

,31 .045 

64 

85 

.04" .07 

18 

2 

.44 .005 

65 

95 

.31" .095 

19 

49 

.45 .04 

66 

91 

.34"" .07 

20 

14 

.37 ^ ‘ .055 

67 

91 

.67"" .07 

21 

4 

.12 .08 

68 

74 

.11"" .06 

22 

9 

.61 .05 

69 

43 

.05"" .055 




70 

45 

.36"" .05 

VB 31 

92 

-.01 .085 

76 

69 

.45*" .05 

32 

92 

.27 .075 

77 

53 

.00"" .055 

33 

88 

.34 ‘‘ .055 

78 

85 

.47"* .06 

34 

98 

.07 .135 

79 

63 

.36"" .05 

35 

83 

.14 .06 

80 

78 

.40"" .055 

36 

60 

.18 .055 

81 

43 

.18"" .055 

37 

95 

.20 .095 

82 

56 

.40"" .05 

38 

92 

.24 .08 

83 

65 

.56"" .045 

39 

97 

-.14 .115 

84 

66 

.38"* .05 

40 

89 

.53 .065 

85 

50 

.52*" .045 

41 

92 

.31 .075 

86 

91 

.63"" .06 

42 

91 

.29 .075 

67 

96 

.45"" .090 

43 

91 

.34 “ .07 

88 

52 

.45** .045 




89 

70 

.41** .05 

44 

97 

.50 .105 

90 

73 

.37** .065 
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In the case of the 30 tasks done by the 240 college grad- 
uates, the correlations vary from .18 to .90. 

Otrue = V*0208-.0059 or .122 (n = 30). 


TABLE 13 

Peemilles Succeeding and CorreijAtions with a Criterion of InteiiDEOT, 

IN THE GaBE of 240 CODl/EGE GRADUATES. 



Task 

Permille 's 

Tti 

«rtl 

oo 

1 

600 

.42 

±: .07 


2 

754 

.38 

“ .08 


3 

775 

.55 

.07 


4 

567 

.57 

.06 


5 

654 

.43 

.07 


6 

458 

.31 

.07 


7 

521 

.18 

.08 


8 

729 

.54 

“ .07 


9 

733 

.24 

.08 


10 

575 

.49 

.06 

AZ 

1 

792 

.39 

.08 


2 

779 

.45 

.08 


3 

642 

.23 

.OS 


4 

467 

.20 

.08 


5 

421 

.33 

** .07 


6 

679 

.30 

.08 


7 

671 

.29 

.08 


8 

642 

.42 

.07 


9 

700 

.43 

.07 


10 

600 

.47 

4 

o 

VA 

61 

671 

.59 

“ .06 

2 

62 

496 

.46 

.07 


63 

775 

.51 

“ .07 


64 

983 

.90 

.09 


65 

808 

.48 

.07 


66 

396 

.42 

.07 


67 

650 

.41 

.07 


68 

292 

.45 

“ .07 


69 

650 

.31 

.08 


70 

192 

.28 

‘‘ .09 
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TABLE 14 

The Corb£i#ations (Bi-8eriai« r) of Each or 99 Beading and VocABxrZiAJtT 
Tasks with Intjcllect (I.E.B. Sel. Bel., Gen. Org.), Gsohped 
According to the Percent Succeeding with 
THE Task. 


5 to 20 

Ttl <Jrt| 

20 to 40 

Ttl ^■‘tl 

40 to 60 

60 to 80 

rtl OTrt, 

80 to 95 

Ttl tTrti 

•37 •!— •Ooo 

.13 ± .05 

.28 =t .045 

.36 It ,045 

A3 ±: .07 

.61 “ .05 

.31 “ .045 

.33 “ .04 

.43 “ .04 

.30 “ .075 


.15 “ .06 

.35 “ .04 

.40 “ .04 

.29 “ .065 


— ,13 “ .06 

.45 “ .04 

,48 “ .04 

.55 “ .04 



— .13 “ .05 

,45 “ .04 

.45 " .045 



.39 “ ,04 

.41 “ ,04 

.53 « .04 



.17 “ .045 

.48 “ .04 

.54 « .04 



.35 “ .045 

.45 " .04 

.52 “ .04 



.22 “ .045 

.46 “ .04 

.28 .045 



— .25 “ .045 

.40 “ .045 

.31 “ .05 



— .43 “ .04 

.49 “ .04 

.39 “ .055 



.48 “ .05 

.30 “ .045 

.24 " .045 



,35 “ .05 

.38 “ .04 

.30 ‘‘ .075 



.05 “ ,055 

.32 “ .015 

.23 " .045 



— .36 “ .05 

.52 “ .04 

.41 .045 



.00 “ .055 

.46 “ .04 

.31 “ .065 



.18 “ ,055 

.49 “ .04 

— .01 “ ,085 



.10 “ .05 

.53 “ .035 

.27 .075 



.52 “ .045 

.39 “ .045 

.34 ,055 



.*3 ” .045 

.35 “ .045 

.14 ” .06 




.32 “ .045 

.24 .08 




.18 “ .055 

.53 .065 




,33 “ .055 

.31 « .075 




.24 " .055 

.29 “ .075 




.52 " .05 

.34 « .07 




.27 “ .055 

.29 « .07 




.29 “ .055 

.39 « .00 




.11 “ .06 

.45 “ .06 




.45 " .05 

— .02 " .07 




.36 " .05 

.48 « .05 




.40 « .055 

.36 “ .07 




.56 “ .045 

.50 « .05 




.38 « .05 

.54 « .06 




.41 « .05 

.04 " .07 




.37 “ .055 

-34 ” .07 
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It is thus clear that only a small part (about one fifth) 
of the variation in the rti’s is due to the limitation to 668 
or 454 or 240 cases. 

Suppose now that we take the mere difficulty of a task 
as a measure of its “intellectual difficulty.” How large an 
error do we make by such neglect of any correction for the 
magnitude of r^ ? Such a procedure is equivalent to treat- 
ing as equal an, at 2 , ots, ot 4 , • * • otn, which after the correc- 
tion would be respectively 


Oi Oi 

Till Tt 2 i 


and SO on. 

rtgi 


Since r^ has a median of about .38 and a mean square vari- 
ation of about .12 for Grade 11 when n is the sigmas 
which we treat as equal and which will in reality not all 


ecfhal vary from about to about 

.oo 


.04 


Oi 

.74 


in about a sixth 


of the tasks Ot will be below 


CTi 

.26 


and in about a sixth of 


them it will be above 


Oi 

.50' 


The sigmas will vary around 


2.63ai with a mean square variation of -SSoi. If the vari- 
ability of an eleventh-grade population is one-fourth of the 
variability of all 17-year-olds, this equals .21a, of all 17>year-oldfl 
or nearly one-thirtieth of the entire range of adult human 
intellect.® 


filt may be well to call attention to the effect of the variability of the 
group upon such correlations as ive have presented. As is well known, a 
correlation of .99 between two measures of intellect for a group composed of a 
random selection of 20>year'OldB will shrink greatly if, by selection for some 
characteristic closely related to intellect, we have a group varying only one- 
fifth OB much as the random group. 

This means of course that the error of a single small task, by its failure to 
utilize all of intellect and its adulteration by factors other than intellect, may 
be a small fraction of the total range of, say, adult human intellecti but a large 
fraction of the range of collegiate intellect. 

The rise in the correlations between score in a single task and intellect 
with wider range of the group used does not impair the validity of anything 
hitherto stated in this chapter as an inference from the correlations found for 
any group. The correction for the error, assuming it to act as a chance 
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Obviously, we do not obtain a very close resemblance 
in ‘intellectual difficulty within a composite of single 
tasks of equal difficulty. This, however. Impairs the value 
of such a composite only slightly, since the variations of 
individuals* intellects in respect to how difficult any given 
single task is to each of them are so great. We use mea- 
sures of intellectual difficulty to measure the level or height 
of individual intellects. If we did have a score of single 
tasks that were of absolutely equal intellectual difficulty 
for, say, eleventh-grade intellects all taken together, they 
would vary greatly in intellectual difficulty for those intel- 
lects taken one at a time, and they would also vary greatly 
in intellectual difficulty in the case of a thousand of these 
eleventh-grade intellects chosen to be all of identical intel- 
lectual level. If, for example, each of such a thousand in- 
tellects gave ten correct responses out of twenty, they would 
not all answer any one task correctly, nor all fail com- 
pletely on any one. The single task does not measure all 
of intellect and nothing but intellect, and so may utilize a 
large fraction of A*s intellect and a small fraction of B’s; 
it may be solved by C largely by factors other than intel- 
lect, while in D there exist non-intellectual factors which 
prevent him from solving it. 

Consequently, a score of single tasks, all with, say, 50% 
of successes, which after correction would be represented 
by twenty values ranging down even to Median grade n — 
2oi grade ii» uud ovcn up to Median gr.«, n -f- 2oi grade ii> will 
nevertheless be a very serviceable composite. 

The two paragraphs preceding the last one are, how- 
ever, really fallacious. Since a single word to be defined 
or sentence to be understood does not have any genuine in- 
tellectual difficulty in the sense of difficulty for all of intel- 
lect and nothing but intellect, we cannot properly attribute 

sampling error, is in terms of the variability of the group for which the eorre- 
lation is found. In absolute units the correction will be the same, the in- 
crease in the variability of the group exactly counterbalancing the effect of the 
increase in the correlation. 
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any amount of intellectual difficulty to it. We should not 
impute purely intellectual difficulty to any save purely in- 
tellectual tasks. A task cannot be purely intellectual when 
it correlates only .40 with intellect in a j;rade population or 
only .80 in an af?e population. We are back at the familiar 
point. We cannot measure the intellectual difficulty of a 
single brief task. 

Moreover, in putting tasks together in a composite, we 
should pay attention to their equality in purely intellectual 
difficulty, if it could be measured, only after much more im- 
portant desiderata had been provided. 

In making up composites of tasks our chief aims are to 
make composites which will correlate highly with intellect 
and which will be of specified difficulty. This means that 
after the first element of the composite has been chosen, the 
merit of the next depends largely upon its partial correla- 
tion with intellect; and after two have been chosen, the 
merit of the next depends largely upon r ,3 , 2 , its partial cor- 
relation with intellect (after elimination of the influence of 
the first and second elements). And so on with the others. 
Getting high partials means getting different aspects of in- 
tellect represented and getting different non-intellectual 
factors counteracted. A moderate amount of wisdom in 
predicting what a given task will do in these respects will 
save much labor in computing rti ^s. 

So far as concerns the first aim, equality in the correla- 
tions (ru’s) is valueless. Among equally difficult tasks we 
would prefer those with the highest correlations and partial 
correlations. 

So far as concerns obtaining a composite of precisely a 
certain specified difficulty, there is no practicable way of 
guaranteeing this beforehand. In practice, however, the 
matter is easy to arrange. We make up each com]>osite 
from tasks of equal difficulty, and then measure the diffi- 
culty of the composites. If we have enough single tasks 
and over a wide enough range, we shall have a great num- 
ber of composites differing progressively by small amounts 
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of difficulty, and can usually find among them one close 
enough to the specified difficulty to serve. If such a one is 
not found, it can be created by combining two neighboring 
composites, taking half of the tasks from each, or four 
tenths from one and six tenths from the other, or whatever 
proportions are likely to give a composite of the desired 
degree of difficulty. 

In the actual selection of elements for a composite, then, 
the main desiderata are to have the percents of successes 
equal and the rti’s either high or with high partial s. The 
equality of the rti ’s is an altogether minor matter.^ 

The facts about variation in the rn^s retain their im- 
portance because we do need to get high correlations with i, 
both for their intrinsic value and because one of the best 
practical ways to get high partial correlations with i is to 
find tasks which measure intellect with different data or 
different operations and still show high total correlations. 
It is a sound rule not to use any single task in a composite 
unless its r,, or the average rti for it or for tasks like it is 
above .30 for a school-grade population of the level for 
which it is intended.® 

Any information about the Tt/s of representative tasks 
is therefore of general value ; and we quote here the results 
obtained by Vincent (’24). Using data furnished by us, 
she measured the correlation between the score attained 
with a single sentence to be completed and the total score 
attained in an intelligence examination of two and a half 
hours, whose reliability coefficient is about .85 for such 

Bo far as concerns boredom from too easy or irritation and discourage- 
ment from too hard single tasks within the same composite, they are due 
chiefly to variations in difficulty, not in intellectual difficulty. They are pre- 
vented chiefly by including tasks which are equally difficult. The tasks should 
also be fairly free from environmental influences; and high total*' correla- 
tions with i are one symptom of this. 

«More precisely, for any two minutes of work we should obtain Vti above 
.40 for Grades 3 to 5, .35 for Grades 6 to 8^ .30 for Grades 9 to 11, and .20 
for Grades 12 to 14. It is harder to get high correlations in the higher grades, 
where the range of intellect may be narrower and where the specialisation of 
the environment is greater. 
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groups as she used. The first group consisted of candidates 
for entrance to college; the second consisted of sixth-grade 
pupils. The examination for the former was the Thorndike 
Intelligence Examination for High School Graduates; that 

TABLE 15 

OVSRLiAPPZNOS AND BI-SeRIALi B's FOR 35 ELEMENTS 


Bi ‘Serial r ’a 

Overlappings with P.E. *s 


o 

d 

.826 

±: .046 

5.2 

.752 

“ .049 

6A 

.726 

« .046 

8.9 

.588 

“ .057 

9.1 

.639 

" .054 

10.0 

.406 

•• .082 

11.7 

.431 

.083 

12.3 

.558 

« .061 

12.4 

.559 

“ .068 

12.5 

.463 

- .068 

12,6 

.559 

- .060 

15.2 

.551 

“ .048 

15.8 

.597 

« .061 

16.5 

.542 

« .041 

18.7 

.665 

“ .031 

20.7 

.520 

.037 

21.2 

.475 

“ .043 

21.5 

.480 

.051 

21.6 

.444 

.044 

22.9 

.427 

.045 

23.3 

.398 

,045 

27.9 

.416 

.053 

28.1 

.352 

« .045 

28.6 

.163 

- .047 

30.4 

.160 

.083 

31.7 

.377 

« .055 

31.8 

.372 

- .081 

31.9 

.343 

« .073 

35.1 

.359 

- .076 

35.6 

.276 

.068 

36.2 

.168 

« .064 

36.7 

.145 

.066 

36.9 

.261 

- .073 

37.7 

.323 

•• .062 

40.9 

.307 

- .083 
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for the latter was made np of Sentence Completions, Arith- 
metical Problems, Vocabulary, Sentence Comprehension, 
and a battery of stock intelligence tests. The correlation 
(using the bi-serial r) varies from .70 or higher to near 0. 

TABLE 16 

The Corsespondence between Success in a Single Small Task and Intel- 
lect, AS Measuked by the Overlapping op the Score in Intellect 
OF Those Failing with the Task Past the Median Score in 
Intellect op Those Succeeding with the Task. 

Compiled prom the Original Data op Vincent. 



Candidates 
for College Fntrance 


Sixth-Grade Pupils 

% of 
Over- 
lapping 

c. 

A. 

Frequencies 
D. All 

0. 

A. 

All 


2 

3 

2 

7 

1 

2 

3 

5-9 

4 

4 

3 

11 

5 

8 

13 

10-14 

5 

11 

14 

30 

6 

12 

18 

15-19 

15 

25 

19 

59 

5 

7 

12 

20-24 

40 

oo 

19 

81 

5 

5 

10 

25-29 

21 

21 

18 

60 

2 

4 

6 

30-34 

19 

17 

17 

53 


4 

4 

35-39 

11 

11 

9 

31 

2 


2 

40-44 

2 

5 

10 

17 

1 

1 

2 

45-49 

1 

1 

3 

5 




50-54 


3 

2 

5 




55-59 

1 

1 


2 




60-64 

1 

1 


2 




65-69 


1 


1 




70-74 



1 

1 




n 

122 

126 

117 

365 

27 

43 

70 

Median 

24 

24 

25 

24 

16% 

14 

15 

Q 




7 



6% 


An overlapping of 25% corresponds to a correlation coefficient of from 
about .30 to about .42 according as the percent succeeding is remote from or 
near to 50. 

An overlapping of 15% corresponds to a correlation coefficient of about .45 
to about .60 according as the percent succeeding is remote from or near to 50. 

The facts for 35 tasks are shown in Table 15. We may use, 
as a measure of the correspondence between score for one 
element and score in intellect measured perfectly or nearly 
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perfectly, the smallness of the overlapping (in the total ex- 
amination scores) of the failures past the median of the 
successes (success and failure referring to the single task 
in question). This ranges from 0 to nearly 50%, the latter 
figure corresponding to a correlation of zero. 

From Vincent’s original data we have compiled Table 
16, which shows the percentages of overlapping for various 
tasks in completing sentences, understanding paragraphs, 
and solving arithmetic problems. 

The overlappings of Table 16 would all be somewhat 
smaller if the measure of intellect were perfect. They 
would be less variable if the nun^ber of individuals used in 
the determinations were larger. This number ranged from 
50 to 499 in the college group (two-thirds of the sentences 
being taken by fewer than 175 individuals) and was either 
about 175, or about 240, or about 375 in the sixth-grade 
group. They would become both smaller and less variable 
in proportion as any mistakes in scoring were eliminated. 
Those for the college group would become less variable if 
the tasks had been done with no limitations of time. The 
sixth-grade groups were allowed to use as much time as 
they wished; the college group were instructed to work as 
fast as they could without making mistakes, and were sub- 
ject to a time limit which was rather generous in the case 
of the completion and the reading tasks, but rather limited 
in the case of the arithmetical tasks. The smaller overlap- 
pings (that is, higher correlations) for the sixth grade 
group are due probably partly to this fact and partly per- 
haps to a greater variability of the sixth-grade group in 
intellect. 

The eleven cases of negative correlation in the college 
group are in some cases due to badly chosen tasks ; in 
others they are due probably to some constant error in the 
scorers ; in others they are due to the chance error attached 
to a determination from a limited number of cases. 
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SUMMAEY 

The sum and substance of this chapter is an emphatic 
conclusion that for every theoretical and practical purpose 
in the measurement of intellectual difficulty, we should use 
collections of tasks rather than single small tasks. We 
ought to measure the difficulty of single tasks ; but we can 
profitably measure intellectual difficulty only in the case of 
composites which contain enough kinds of tasks to repre- 
sent a fair sampling of all of intellect as it operates at that 
level, and enough tasks to make the error closely the same 
for any one composite as for any other with which we wish 
to compare it in respect of difficulty. 

When such composite tasks are attained we can infer 
the difficulty values in terms of oi from the values in terms 
of oti, ot2, 0*3, etc., or, since oti, at2, ots, etc., are closely 
equal, we can use Ot as the unit. 



CHAPTER V 


The Measurement of the iNTELUBcyruAL Difficulty of 
Tasks by a Consensus of Expert Opinion 

No one doubts that a certain validity attaches to human 
judgments of the difficulty of intellectual tasks, — ^that, for 
example, it is harder to find an opposite of “government’^ 
beginning with “a” than an opposite of “below” beginning 
with “a,” or to answer correctly, “How many quarters of 
a quarter equal half of a half?” than to answer correctly, 
“How many cents are three cents and one cent?” 

If a thousand psychologists or others who are ac- 
quainted with intellectual tasks are required to state which 
of two tasks is harder, the amount of agreement is a mea- 
sure, or at least a symptom, of the magnitude of the real 
difference. If 910 of the thousand rank A as harder than 
K, whereas only 510 of the thousand rank B as harder than 
K, the difference A-K will be supposed by all sensible per- 
sons to be greater than the difference B-K, except for 
tasks in respect of which the thousand suffer from some 
illusion or constant error. 

Given the truth of certain assumptions about the judges 
and the process of judging, the magnitude of the real dif- 
ference may be determined from the percent of judges 
discerning it. These assumptions have been used as a pro- 
visional way to determine the magnitudes of differences in 
the general merit of handwriting and drawings by Thorn- 
dike [’10 and ’13] and Kline and Carey [’23] ; the general 
merit of compositions by Hillegas [ ’12] ; and the beauty of 
designs by Thorndike [’16]. They have been used widely 
by HoUingworth, Strong and others in measuring various 
features of advertisements. 

This method of deriving units of measure is more appro- 
priate in the case of certain esthetic and ethical values. 
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where to be more beautiful (amusing, sublime, refined, rev- 
erent, patriotic) is to be thought so, than in the case of in- 
tellectual difficulty, but it has obvious advantages in econ- 
omy and ease of application, and, at the worst, it utilizes 
human judgments without exaggerating their intrinsic 
errors and without introducing any new errors. 

What the intrinsic errors of human judgments of the 
difficulty of intellectual tasks are has not been known be- 
cause hitherto there has been no extended or systematic 
study of such judgments, and no criterion against which to 
check their validity and precision. 

THE EXPERIMENTS 

We have carried out two experiments with these impres- 
sionistic judgments of the difficulty of tasks. The first, 
which was reported in the J ournal of Educational Research , 
February, 1924, [Thorndike, Bregman and Cobb, *24] used 
a hundred tasks as the material to be judged, and forty stu- 
dents of psychology and education as the judges. The raw 
correlation of the ranking for difficulty by the consensus 
with the ranking by the percentages of a group succeeding 
with the respective task elements was .88. 

The second experiment used some twelve hundred tasks 
and twenty sets of judgments, these being made by Dr. E. 
M. Bailor, Dr. E. O. Bregman (2), M. V. Cobb (2), Dr. A. 
I. Gates, Z. P. Miner, Dr, B. Pintner, E. E. Eobinson (2), 
G. J. Euger, Dr. L. S. Tlollingworth, Dr. Godfrey Thomson, 
Dr. L. M. Vincent (2), J. W. Tilton, Dr. B. D. Wood, Ella 
Woodyard (2), and E. E. Thorndike (2). 

The number 2 in parenthesis means that the person in 
question made two sets of judgments. The instructions for 
the ratings and a few sample tasks including some near 
both extremes are quoted below. 

INSTRUCTIONS FOR GRADING 

Each slip is a task in Arithmetic, Sentence Completion, Vocabulary, 
Directions, Beading, Information, or giving Opposites. The nature of each 
task will be apparent, if you remember tliat: 

11 
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1. A single word preceded by a letter means that the task is to give an 
opposite to the word beginning with that letter, ^*b .white'' requires 
** black" as the response. 

2. A word followed by 5 other words means that the task is to select that 
one of the five which means most nearly the same as the first word, 

powerful holy strong during sad old" requires strong" as the re- 
sponse. 

3. A word followed by 4 or 5 pictures means that the task is to select 
that one of the pictures which best fits the word. 

4. If the task is preceded by the word "oral," the task is not to be shown 
to the person in print, but put to him clearly orally, and repeated once, and a 
second time if he desires. 

5. If the task is not preceded by the word ‘^oral," the person doing the 
task is supposed to have the opportunity to read and re-read it. If he has 
difficulty in reading, he is supposed to have the task stated fo him orally in 
whole or in part as often as he wishes. 

The tasks are to be rated in 200 or more groups, in respect of their intel- 
lectual difficulty, for a group of persons twenty years old brought up in the 
United States, with an opportunity to go to school for at least 7 years, unless 
they were so dull as to be unable to learn at school. At one end will be the 
tasks which you think only the best intellects would do correctly; at the other 
end will be those which all save the lowest imbeciles would do correctly. 

You should assume that the general nature of the task of giving an oppo- 
site, or of completing a sentence, or of selecting the word most nearly of the 
same meaning, has been stated in very simple language and illustrated by five 
easy samples, and that the tasks of any one sort are given at one time and in 
an order beginning with the easiest. 

In all ratings pay no attention to the possibility of chance successes. 
Think of the difficulty of the task in every case as the difficulty of succeeding 
with it by real knowledge or ability. 

In about one case out of 200 there was an omitted or am- 
biguous rating. To simplify later computations, an esti- 
mate was made of the probable intent of the judge in such 
cases, by consideration of his ratings of four tasks of ap- 
proximately the same sort. 

The basis for the judgments doubtless varies from one 
judge to another and from one task to another for the same 
judge, and for the same task for the same judge at different 
times. It would be interesting and perhaps valuable to dis- 
cover what qualities in a task and w^hat facts or fancies 
about it make any given judge regard it as hard. We shall, 
however, limit our inquiry to the ratings themselves re- 
gardless of how they were caused. 
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THE EATINGS 

The ratings were combined by simple addition, the re- 
sult being a series of arbitrary numbers from 32 to over 
3,600 which represent accurately enough for all our pur- 
poses an order of difficulty by the consensus. Its statistical 
reliability is fairly high. The sum of the ratings by ten of 
the judges (J3r., Mi., Ro. (2), Thom., Thor. (2), Vi. and 
Wo. (2)) corresponds closely with the sum of the ratings by 
the other ten. The facts appear in Table 17. 

The reliability is about the same for any one sort of 
task, such as sentence completion, or arithmetical prob- 
lem or word knowledge, as for the entire series. That is, 
the judges agreed about as closely when they compared two 
tasks of different sorts as when they compared two tasks 
of the same sort. 

The correlations between the two sums of ten are as fol- 
lows: 


Completion tasks 

.973 

Arithmetic tasks 

.988 

Vocabulary tasks _ 

.954 

Directions tasks 

.996 

Information tasks 

.979 

Opposite tasks 

.978 


The average of the six is .978. The correlation when all 
are mixed together is .984. 

This material is unsuitable for the computation of co- 
efficients of correlation, the distributions being of very ir- 
regular form. The correlations given above are used only 
as rough indicators of the closeness of agreement between 
the two groups of ten judges. 

The mean square error and the median or probable 
error of the sum of the twenty ratings for any task are as 
shown in Table 18.^ The error varies, increasing in general 

1 These measures of unreliability are computed from the mean square devia- 
tions of the differences between the sum of the ratings of the ffrst ten and 
the siun of the ratings of the second ten judges. The mean square error for 

a sum of ten equals'-- — rr^Cdiff,. The mean square error for the average 
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with the difl&culty of the tasks, but also decreasing at the 
two extremes of the set of tasks used. On the average, it is 
about one thirtieth of the difference between task I and 
task H, shown at the bottom of Table 18, for the mean 
square error, and about one forty-fifth thereof for the prob- 
able error. 

TABLE 18 

The Probable Divergence or a DimcuLTY Rating by 20 Experts from the 
Average of an Infinite Number of DimctrLTY Ratings of the Task, 
(Each Rating Being the Average of the Ratings of 20 Experts). 



The unit being thej 
same as that of the 
difficulty ratings by 
the 20 experts 

The unit being one 
hundredth of the 
diff6rence2 between 
Level A and 
Level O 

The unit being one 
hundredth of the 
difference between 
difficulty rating for 
Task 1 and Task 11 

Tasks rated under 

S.D. 

P.E. 

S.D. 

P.E. 

S.D. 

P.E. 

400 (approx.) 

44 

29 

1.5 

1.0 

1.3 

.8 

400 to 799 

81 

55 

2.8 

1.9 

2.3 

1.6 

800“ 1199 

97 

66 

3.3 

2.2 

2.8 

1.9 

1200 “ lf>99 

99 

67 

3.4 

2.3 

2.9 

1.9 

1600“ 1999 

129 

87 

3.7 

2.5 

3.2 

2.1 

2000 “ 2399 

no 

74 

4.4 

3.0 

3.7 

2.5 

2400“ 2799 

117 

79 

4.0 

2.7 

3.4 

2.3 

2800“ 3199 

108 

73 

3.7 

2.5 

3.1 

2.1 

3200 “3599 

86 

58 

2.9 

2.0 

2.4 

1.7 


1. Hold up your hand. 

2. Show me yo\ir nose. Put your finger on your nose. 

3. Show mo your mouth. Put your finger on your mouth. 


of two sums of ten equals — x 

V2 


V2 


— X CTdiff.- 


Since, however, we are using 


the sum of twenty in place of the average of two sums of ten, our numbers 
are all twice as large as they would be for the average of two sums of ten. 
That is, the mean square error for a sum of twenty equals: 


2 X — ^ X — - XcTdiff. or simply cTdift.. 

2 Level A is the ability of adults of mental age a little under 36 months, and 
80 with I.Q.’6 of about 20. 

Level O is approximately the ability of the average graduate of American 
eoUeges of high requirements. 
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n 

1. Bead this and then write the answers. Bead it again if you need to, 

COLERIDGE 

I see thee pine like her in golden story 
Who, in her prison, woke and saw, one day, 

The gates thrown open — saw the sunbeams play 
With only a web ^tween her and summer's glory; 

Who, when the web— so frail, so transitory, 

It broke before her breath — had fallen away, 

Saw other webs and others rise for aye, 

Which kept her prisoned till her hair was hoary. 

Those songs half -sung that yet were all divine — 

That woke Romance, the queen, to reign afresh — 

Had been but preludes from that lyre of thine. 

Could thy rare spirit's wings have pierced the mesh 
Spun by the wizard who compels the flesh, 

But lets the poet see how heav'n can shine. 

Copy the first word of the line which implies there had not been a con- 
tinuous stream of like songs. 


2. Supply the missing words to make this a true and sensible sentence. 

Speech, gesture and form of human 

action are in run resolvable 

contraction. 

3. Arrange these numbers and signs to form a true equation. 

2/3 2 3 15 15 = - - X 

So much of these unreliabilities as is due to the small 
number of judges can be reduced to any desired extent by 
increasing the number of judges. The crude summations 
of ranks can also be replaced by more precise and refined 
uses of the differences between the rankings for any two 
tasks. The general value of the method can, however, be 
studied well enough for our purposes with the sums of the 
twenty ranks as they stand. 

The meaning of these sums of the twenty ranks in terms 
of the percentage of the judges who judge the direction of 
the difference correctly may be realized from the following 
facts : 
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Taking 618 pairs of tasks at random from those pairs 
which differ in the ‘‘sum of the twenty” by approximately 
100 (95 to 105), we find that, in 263, eleven and a half or 
fewer of the twenty judges® judged correctly ; in 114, twelve 
judged correctly; and in 241, twelve and a half or more 
judged correctly. A difference of 100 in the “sum of the 
twenty ranks” thus corresponds to a percentage of judges 
a little under 60. 

Taking 853 pairs of tasks at random from those pairs 
which differ in the “sum of the twenty” by approximately 
200 (195 to 205), we find that, in 404, thirteen or fewer of 
the judges judged correctly; in 49, thirteen and a half 
judged correctly; in 400, fourteen or more judged cor- 
rectly. A difference of 200 in the sum of the twenty ranks 
thus corresponds almost exactly to a percentage of 67f . 

A percentage of 67^ correct means a difference of .673 
times the median deviation of the judges in ability to judge 
the intellectual difficulty of tasks, and 60% means a differ- 
ence of .375 times it. So we may regard the median devia- 
tion (or difference observable by 75% of these judges) as a 
bit over 300 in the units of the “Sums of twenty.”* The 
entire range is thus only about twelve times the amount of 
difference which 75% of these judges recognize, which 
means, of course, that our judgments of the intellectual dif- 
ficulty of tasks are not acute. 

THE VAIJDITY OF THE CONSENSUS 

The important matter is, of course, the validity of the 
consensus, — its correspondence with intellectual difficulty 
when that is objectively determined. How far we have a 
right to use a consensus of expert opinion to measure the 
difficulty of a task depends upon the freedom of the con- 
sensus from systematic or “constant” errors, such as a 

3 When any judge assigned the same rank to the two tasks which the twenty 
put as 100 apart, he was scored as half right and half wrong. 

^ It will be a little less than that at the two extremes, and more than that in 
the middle, the agreement of the judges being closer for a difference of 100 or 
200 at the extremes than for the same numerical difference in the middle ranges* 
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tendency to over-estimate the intellectual difficulty of com- 
pletions in comparison with arithmetical problems, or to 
underestimate differences in intellectual difficulty at the 
hard end of the series, or to fail to see differences over a 
certain range of the scale. 

We can determine intellectual difficulty objectively for 
certain composite tasks. Thus, for very dull adults the re- 
spective intellectual difficulties of the composites A, B, C, 
and D are determined by the facts that 159, 87, 23, and 1 out 
of 180 such dull adults succeed w^ith these composites. That 
the composites measure Intellect CAVD is elsewhere 
proved. From the form of distribution of the group of 180, 
the differences betw^een A, B, C, and D in intellectual diffi- 
culty may be found. 

If now we examine the average consensus estimate of 
difficulty of the forty elements of composite A, and simi- 
larly for B, C, and D, we may easily compute the average 
differences between A, B, C, and D in intellectual difficulty 
as estimated by the consensus. The closeness of correspon- 
dence of the objectively and subjectively determined sets 
of differences in intellectual difficulty may be measured in 
various ways. A similar procedure can be carried out for 
composites used with a group of college graduates, or for 
any other set of composites, whose intellectual difficulty is 
objectively measured in suitable units. In proportion as 
the consensus agrees closely with the objective results in 
the case of composites where we have such objective re- 
sults, we can trust the consensus® in the case of tasks where 
we lack objective results. It is therefore of great impor- 
tance to inquire how close this correspondence is, how free 
the consensus is from errors other than the variable errors 
due to the small number of experts. 

It will be useful to state in a summary manner the out- 
come of the inquiry before presenting its details. 

Such a consensus, even from a thousand experts, will not 
be trustworthy throughout. It will make blunders, suffer 

B Subject to due couBideratiou of its variable errors, and within the range 
of difS^eulty and sort of task where it has been proved valid. 
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from “constant errors/* some of them regrettably large. 
We shall see, for example, that it overestimates the diffi- 
culty of easy sentence-completions in comparison to that of 
directions or vocabulary or arithmetic tests, and that it 
fails to observe genuine' differences in difficulty within a 
rather wide range of vocabulary tests. If composite tasks 
were made up on the basis of its estimates of the intellec- 
tual difficulty of tasks, the difficulty of these composites as 
composites would need to be carefully measured objectively. 

The consensus would, on the other hand, often be near 
the truth and rarely be greatly in error. There will be a 
substantial correlation with objectively determined results. 
If such a consensus alone had been used to estimate the dif- 
ficulty of single tasks, and CAVD composites of forty had 
been constructed on the basis of its estimates, they would 
have been serviceable composites, forming a gradation in 
intellectual difficulty, and containing in any one composite 
few single tasks which would appear puerile on the one 
hand or mystifying on the other to the individuals who 
could succeed with half of the tasks in that composite. 
After being evaluated as composites by objective methods, 
these composites would be not much inferior to those which 
we have constructed at enormous cost of time and labor 
spent in experimentation. Consequently, the use of esti- 
mates by a suitable consensus may well replace measure- 
ments of the percents succeeding in the case of single tasks, 
in the preliminary work of making composite tasks. 

The evidence that the consensus is in certain respects 
definitely wrong is as follows : 

We have four composites each made up of ten sentence- 
completions ; four, each made up of ten arithmetical tasks ; 
four, each made up of ten vocabulary tasks; four, each 
made up of ten directions, and four, each made up of ten 
information tasks. Each of 180 very dull adults was tested 
with each of these twenty 10-composite tasks, which we 
shall designate hereafter as 
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c 

A, 

B, 

c, 

and D 

A 

<4 

44 

44 

44 

44 

V 

44 

44 

44 

44 

44 

D 

4 4 

44 

44 

4 4 

44 

I 

44 

44 

44 

44 

44 


The results were as shown in Table 19, the 180 individuals 
being divided into two groups, of 100 at one institution, 
and 80 at another. 

TABLE 19. 

Measures or the Difficulty of 10-Composite Tasks. 


By Experiment By the Consensus 




Percent 
Suec coding 

Distances from 
the Median. 

In terms of aio 

Median Sum of 

20 Expert Ratings 

No. of 
Tasks Bated 


n = 

100 

80 

n = 100 80 



c 

A 

84 

82% 

-1.19 

-1.13 

800 

8 


B 

65 

56 

- .51 

- .20 

830 

2 


o 

35 

27% 

+ .45 

+ .68 

970 

4 


D 

3 

0 

+ 1.59 

high 

1023 

4 

A 

A 

69 

80 

- .65 

-1.03 

309 

3 


B 

45 

49 

+ .15 

+ .03 

536 

2 


C 

15 

21 

+ 1.05 

+ .87 

458 

2 


D 

5 

5 

+ 1.47 

+ 1.47 

858 

3 

V 

A 

80 

81 

-1.03 

-1.07 

292 

9 


B 

49 

.*57% 

+ .03 

- .25 

562 

9 


C 

14 

19 

+ 1.09 

+ .93 

925 

7 


D 

1 

5 

+ 1.78 

+ 1.47 

848 

6 

D 

A 

90 

86 

-1.45 

-1.27 

not over 300 



B 

45 

67% 

+ .15 

- .59 




C 

19 

27% 

+ .93 

+ .68 

529 

5 


D 

12 

14 

+ 1.16 

+ 1.09 

668 

10 

I 

A 

76 

83% 

- .89 

-1.17 

341 

7 


B 

51 

59% 

- .03 

- .32 

618 

4 


C 

23 

21% 

+ .81 

+ .86 

733 

7 


D 

3 

4 

+ 1.59 

+ 1.53 

792 

6 


In 

the 

same 

table 

are shown the median 

summation 

scores 

for the single tasks in 

each of these composites so 
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far as they were included in the 1200 rated, and the ntmaber 
so included.® 

There is an obvious ‘‘constant error” in the direction 
of overestimating the difficulty of the sentence-comple- 
tions, especially the easier ones. To be in line with the 
other tasks, the figures for them should be, respectively, 
about 500, 300, 350, and 200 lower than they are. There is 
a failure to distinguish the Arithmetic B’s from the Arith- 
metic C’s. There is a similar failure with the Vocabulary 
C’s and D’s. 


TABLE 20. 

DiFFEREKCBS in DlITICTJIiTY OF VARIOUS COMPOSITE TASKS AND OF THE 

Median Sums of 20 Expert Eatings of the Singue Tasks of These 
Composites Which Were Bated, Each Difference Is Ex- 
pressed AS 4. Percent or the Difference Between the 
A AND THE D Composite of Its Kind. 



Differences in Difficulty 


DlsagrecmentB 



By Experiment 
n = 100 80 

By the 
Consensus 

100 

M'ifli 

80 

100 80 
with with 

Consensus Consensus 

OB~C A 

24 

30* 

10 

6 

34 

20 

OO-CB 

35 

28* 

65 

7 

30 

37 

CD-CG 

41 

42* 

25 

1 

16 

17 

AB^A A 

38 

42 

41 

4 

3 

1 

AC-AB 

42 

34 

•~14 

8 

56 

48 

AD-AC 

20 

24 

73 

4 

53 

49 

VB-V A 

38 

32 

49 

6 

11 

17 

VCV-VB 

38 

46 

65 

8 

27 

19 

VD-VC 

25 

21 

-14 

4 

39 

35 

IB-I A 

35 

31 

61 Vj 

4 

26% 

30% 

lO-IB 

34 

44 

25 

10 

8% 

18% 

ID-IG 

31 

2o 

13 

6 

18 

12 

Sum of disagreements 



68 

322 

304 


* The difficulty of C D is estimated as + 2.00. 


^ The probable errors of these medians will be approximately 50 when n = 2^ 
40 when n = 3, 35 when n = 4, 31 when n = 5, 28% when n=:6, 26% when n. = 7, 
25 when n = 8^ 23 when n = 9^ and 22 when n =: 10. 
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If we list the differences between the A and the B, the 
B and the C, and the C and the D tasks and express each as 
a percent of the difference between the A and the D tasks, 
we can observe more readily how closely the consensus par- 
allels the objective results, when the kind of task is kept 
constant. The disagreement between the consensus and 

TABLE 21. 

Fobm or Distribution Used in the CaIjCulations op Tables 19 and 20. 
Belative Frequencies at Equal Successive Intervals. 


Interval 


Frequency 


0 

to 

.99 

5.5 

1 


1.99 

5.6 

2 


2.99 

11 

3 

€ € 

3.99 

11 


t t 

4.99 

17 

5 

i ( 

5.99 

17 

6 

t € 

6.99 

22 

7 

i € 

7.99 

28 

8 

i t 

S.99 

33 

9 

t f 

9.99 

33 

10 

it 

10.99 

34 

11 

it 

11.99 

39 

12 

a 

12.99 

38 

13 

1 1 

13.99 

39 

14 

1 1 

14.99 

39 

15 

< i 

15.99 

39 

16 

1 1 

16.99 

39 

17 

i € 

17.99 

39 

18 

1 1 

18.99 

44 

19 

i i 

19.99 

45 

20 

i i 

20.99 

44 

21 

if 

21.99 

45 

22 

t i 

22.99 

44 

23 

i f 

23.99 

44 

24 

a 

24.99 

45 

25 

1 ( 

25.99 

44.5 

26 

a 

26,99 

38.5 

27 

i i 

27.99 

33.7 

28 

i € 

28.99 

27.8 

29 

it 

29.99 

22.3 

30 

t i 

30.99 

16.6 

31 

i t 

31.99 

12.2 

32 

it 

32.99 

5.5 
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the result of either experiment is about four times as great 
as the disagreement between the results of the two experi- 
ments. The facts are shown in Table 20. 

In the calculations of Table 19 and Table 20, the form 
of distribution of these low imbeciles is taken to be that 
shown in Table 21. We ask the reader to take the validity 
of this form of distribution on faith for the present, or to 
turn to Appendix VI and examine the facts given in con- 
nection with its derivation there. To show that the present 
conclusion does not depend for its validity upon the par- 
ticular form of distribution used, we have carried through 
the computations supposing it to be Form A (the “nor- 
mal** form) and supposing it to be a rectangle. The results 
appear in Table 22 and Table 23. 

The resemblances may also be measured crudely by cor- 
relation coefficients, after first expressing the estimates of 
difficulty as a rank order. The correlations’’ using 

__ . 

P — ^ — ~r 2 — T\ are 
n(n® — 1) 

.97 for the experiment with 100 with the experiment with 
80; 

.62 for the experiment with 100 with the consensus ; 

.70 for the experiment with 80 with the consensus. 

We have records from 240 college graduates and from 
189 candidates for college entrance with the composites of 
ten tasks listed in Table 24. Wc have also computed the 
medians of the 20-expert sums of ratings of such tasks in 
each composite as were rated by the experts- These and 
the measures of difficulty from the experiments with the 240 
and the 189 are entered in Table 24. In this case the form 
of distribution of intellect in the 189 is known to be ap- 

7 We may here use p for r without transmuting, since the form of distribu- 
tion of these twenty composites in respect of difficulty is probably better repre- 
sented by a rectangle than by a surface of Form A. If transmuted, all would 
be a trifle higher. In the ranks for the consensus, D A is put as 1 and D B as 
5 on the basis of the ratings by the consensus of certain tasks closely resembling 
the tasks of D A and D B. 
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proximately of Form A ; and that of the 240 may be treated 
as such with no damage to the present argument.* 

A general inspection of Table 24 reveals notable irregu- 
larities in the measures by the consensus, as when A O is 
rated 178 points easier than AN, and D5 is rated 242 
points easier than D4^. The consensus seems to fail to dis- 
criminate well in general among the D (reading) tasks. 

TABLE 22. 

Measures of Difficulty if the Form of Distribution Assumed Is Form A 
OR A Bectangle. Distance from C.T. in Terms of a and of Q/25. 



Percent 

Succeeding 

In terms of cr 
assuming 

Form A 

In terms of Q/25 
assuming a 
rectangle 


100 

80 

100 

80 

100 

80 

0 A 

84 

82% 

- .995 

- .935 

-34 

-32% 

B 

65 

56 

- .385 

- .158 

-15 

- 6 

C 

35 

27% 

+ .385 

+ .598 

+ 15 

+ 22% 

D 

3 

0 

+ 1.881 

high 

+ 47 

high 

A A 

69 

80 

- .496 

- .842 

-19 

-30 

B 

45 

49 

+ .126 

+ .031 

5 

+ 1 

0 

15 

21 

+ 1.036 

+ .798 

+ 35 

+ 29 

D 

5 

5 

+ 1.645 

+ 1.645 

+ 45 

+ 45 

V A 

80 

81 

- .842 

-- .883 

-30 

-31 

B 

49 

57% 

+ .050 

-- .189 

+ 1 

- 7% 

0 

14 

19 

+ 1.080 

.887 

+ 36 

+ 31 

D 

I 

5 

+ 2.326 

+ 1.645 

+ 49 

+ 45 

D A 

90 

86 

- 1,282 

- 1.092 

-40 

-36 

B 

45 

67% 

+ .126 

- .454 

+ 5 

-17% 

0 

19 

27% 

+ .878 

+ .598 

+ 31 

+ 22% 

D 

12 

14 

+ 1.175 

+ 1,092 

+ 38 

+ 36 

I A 

76 

83% 

- ,706 

- .974 

-26 

-33% 

B 

51 

59% 

- .025 

- .240 

- 1 

- 9% 

O 

23 

21% 

+ .739 

+ .789 

+ 27 

+ 28% 

D 

3 

4 

+ 1.881 

+ 1.761 

+ 47 

+ 46 


B7 the best treatment which we are able to make of the available evi- 
dence, the form of distribution of level of intellect in the 240 college graduates 
diverges from Form A onlj in the manner and to the extent shown in Table 
165 of Appendix VI. 
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6ZD* 


with the rank 


The correlations, nsing p = l— 

n(n® — 1) 

orders, are .94 for the 240 experiment with the 189 experi- 
ment, .79 and .72 foi the correlations between experimental 


TABLE 24. 

DirprcoiiTT or Twelve Composites by the Besttlts with 240 College Grabit- 
AXES AND 189 Candidates roa College Entrance, in Distances + and - 
raoM the Median fob the 240, m Tyems of the a of the Com- 
posite Concerned. Also the Median P^ tings by the Consen- 
sus OF Such Tasks in Each Composite As Were Bated. 


Estimated 



By the 

240 

Difficulty 

By the 
189 

By the con- 
sensus : 

Median 

N 

Probable 
Error of the 
Consensus 
Median 

ON 

-1.47 

-1.29 

3084 


6 

35 

o 

- .87 

- .77 

3279 


0 

6C 

P 

- .29 

- .00 

3482 


3 

50 

Q 

+ .21 

.86 

3314 


1 

95 

A N 

-2.03 

-2.07 

28.55 


6 

35 

o 

-1.23 

-1.18 

3047 


3 

50 

p 

- .92 

- .67 

2869 


4 

42 

Q 

— .45 

- .36 

3338 


3 

50 

D4% 

-1.79 

-2.07 

3191 


4 

42 

5 

-1.08 

-1.54 

2949 


3 

50 

6 

- .64 

-1.07 

3258 


3 

50 

7 

+ .30 

+ .16 

3291 


1 

85 




Differences 


Disagreements 


By the 

By the 

By the 

240-. 

240-- 

189- 


240 

189 Consensus 

189 

Cons» 

Cods. 

C O-N 

36 

24 

85 

12 

49 

61 

P-O 

34 

36 

88 

2 

54 

52 

Q-P 

30 

40 

- 73 

10 

103 

113 

A O-N 

50 V 2 

52 

39% 


11 

12% 

p~o 

19% 

30 

- 36% 

10% 

56 

66% 

Q-P 

30 

18 

97 

12 

67 

79 

D 5-4% 34 

24 

-242 

10 

276 

266 

6-5 

21 

21 

309 

0 

288 

288 

7-6 

45 

55 

33 

10 

12 

22 

Sum of disagreements 


68 

916 

960 
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results and the consensus. A treatment of the differences 
in terms of percents like that of Table 20 is presented in the 
lower half of Table 24. The disagreements between ex- 
periment and consensus are 13 or 14 times as large as the 
disagreements between the two experiments. The disagree- 
ments between experiment and consensus would, however, 
be reduced if we had ratings of five or six tasks instead of 
one in the C Q and D7 composites. 

We have extensive experiments with the vocabulary 
composites la, 2a, 3a, 4a, 5a, 6a, and 7a with pupils in 
Grades 9, 10, and 11 ; and four of the tasks of each of these 
composites were rated by the consensus. The essential 
facts appear in Table 25. The consensus is badly in error 
in putting la much too low, or 2a much too high, or in doing 
both.® It also crowds 4a, 5a, and 6a close together, failing 
to distinguish fully the large differences which exist be- 
tween these. It makes the sum of the differences between 
la and 2a and between 6a and 7a nearly four times as large 
as the entire difference between 2a and 6a, though the ex- 
perimental results make the latter nearly twice the former. 
The disagreements between the consensus and any experi- 
ment are about seven times as large as the disagreements 
between any one experiment and any other. In spite of 
these notable errors, there remains a general correspon- 
dence between consensus estimates and experimental re- 
sults. The rank-order correlation is indeed almost unity. 
Table 25 showing no reversals. 


0 It would not be fair to make this statement on the basis of the facts 
of Table 25 alone^ since both V la and V 2a are so easy for pupils in 
Grade 9 and above that results from these grades are not suitable to measure 


the difficulty of either at all accurately. We have evidence from a group of 
#00 pupils in Grade 5%, however, to the effect that the difference between 
la and V 2a is less than the difference between V 2a and V 3a. 

percents correct are: 99.5, 89.5, and 58.0 Taken at their face value, 
e give differences of ,88o for la to 2a and 1.50a for 2a to 3a- The .88 
be too small because the one pupil in 200 who failed to get five of the 10 
in la right may have been extremely careless. Very, very low percents 
course unreliable for n = 200, for many reasons. It is extremely un- 
however, that the true a value for V la will be below —3.20, bo as 
the difference between la and 2a actually greater than the difference 
2a and 3a. 
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We have also very extensive experiments with the D 
(reading) composites 1 to 7, the results from which are pre- 
sented in Table 26. Unfortunately, the median consensus 
ratings for composites 1 and 7 are from only one task 
each. Even after liberal allowance for the large probable 
errors of these medians, there is a clear failure of the con- 

TABLE 25. 

Measures op Difficulty. 




Grade 9 

11 = 1041 

Grade 10 
n=700 

Grade 11 
n=752 

Consensus 

Medians 

n 


V la 

n = 10 

- 2.457 

- 2.457 

- 3.090 

2139 

4 


«* 2a 

1 1 

- 2.576 

-2.366 

- 3.090 

2622 

1 i 


3a 

€ 1 

-1.3C6 

- 1.468 

- 1.995 

2679 

1 1 


“ 4a 

i { 

- .719 

- .990 

- 1.483 

2787 

£ £ 


** 5a 

t i 

H- .068 

- .264 

- .845 

2809 

£ £ 


6a 

t € 

+ .904 

+ .527 

+ .050 

2854 

£ £ 


7a 

li 

+ 2.120 

+ 1.866 

+ 1.259 

3227 

£ £ 





Differences 




2a— la 


.119 

.091 

.000 

483 



3ar*2a 


1.210 

.898 

1.095 

57 



4a-3a 


.647 

.478 

.512 

108 



5a--4a 


.787 

.726 

.638 

22 



6a--5a 


.836 

.791 

.895 

45 



7a-6a 


1.216 

1.339 

1.209 

373 





Differences 

Divided hy 

the 7a-la Difference 


2a— la 


.026 

.021 

.000 

.444 



3a-2a 


.264 

.208 

.252 

.052 



4a-3a 


.141 

.111 

.118 

.099 



5a— 4 a 


.172 

.168 

.147 

.020 



6a— 5a 


.183 

.183 

.206 

.041 



7a—6a 


.266 

.310 

.278 

.342 






Discrepancies 






9-10 

9-11 

10-11 

9— Con. 

lO-Con. 

11— Con. 

2a— la 


.005 

.026 

.021 

.418 

.423 

.444 

8a— 2a 


.056 

.012 

.044 

.212 

.156 

.200 

4a— 3 a 


.030 

.023 

.007 

.042 

.012 

.019 

5a— 4a 


.004 

.025 

.021 

.125 

.148 

.127 

6ar-5a 


.000 

.023 

.023 

•142 

.142 

.165 

7a— €a 


.044 

.012 

.032 

.076 

.032 

.064 

Sum 


.139 

.121 

.148 

1.042 

.913 

1.019 
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Bensns to distinguish differences in difficulty accurately 
amongst these reading tasks. The range from D1 to D7 
represents the range from what two thirds of pupils in 
Grade 6 can do to what not one in twenty-five high-school 
seniors can do. By the experimental results, it is clear that 

TABIjE 26. 

Measures of Difficui/ty. 



Grade 30 
n = 1185 

Grade 11 
n = 1053 

Grade 12 
n = 742 

Consensus 

n 

D 1 

- 2.409 

-2.878 

- 2.652 

2713 

1 

D2 

- 1.398 

- 1.695 

- 1.812 

2743 

4 

D3 

- .690 

- .999 

-1.170 

3118 

2 

D4 

- .055 

- .333 

- .516 

3177 

5 

D5 

+ .542 

+ .306 

- .065 

2949 

3 

D6 

+ 1.243 

•f .966 

+ .824 

3255 

4 

D7 

4 2.170 

+ 1.896 

+ 1.774 

3291 

1 




Differences 



D2-1 

1.011 

1.183 

.840 

30 


D3-2 

.708 

.696 

.642 

375 


D4-3 

.635 

.664 

.654 

59 


D5-4 

.597 

.641 

,451 

-228 


D6-5 

.701 

.660 

.889 

306 


D 7-6 

.927 

.930 

.950 

36 



Differences Expressed as Fractions of the D7—D1 Difference 


O 2 — 1 

.221 

.248 

.190 

.052 



D 3-2 

.155 

.146 

.145 

.649 



D4-3 

.139 

.139 

.148 

.102 



D5-4 

.130 

.134 

.102 

-.394 



D 6-5 

.153 

.138 

.201 

.529 



D7-6 

.202 

.195 

.214 

.062 






Disagreements 





10-11 

10-12 

11-12 

10— Con. 

11-Con. 

12-Con, 

D2-1 

.027 

.031 

.058 

.169 

.196 

.138 

D3-2 

.009 

.010 

.001 

.494 

.503 

.504 

D4-3 

.000 

.009 

.009 

.037 

.037 

.046 

D5-4 

.004 

.028 

.032 

.524 

.528 

.496 

D6-5 

.015 

.048 

.063 

.376 

.391 

.328 

D7-6 

.007 

.012 

.019 

.140 

.133 

.152 

Sum 

.062 

.138 

.182 

1.740 

1.788 

1.664 
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each step is a substantial increase in difficulty. But by the 
consensus we have great irregularities and reversals, and 
the differences 1 to 2 and 6 to 7, which should be the largest, 
are specially small (if we trust the single- task medians). 

The comparisons of the last 10 pages are all subject to 
the criticism that the experimental results do not measure 
the truly intellectual difficulty of the composites in ques- 
tion, but rather their difficulty for whatever ability each 
involves, and that if by a miracle we could know how well 
people would succeed with these composites if each individ- 
ual could use all of his intellect and nothing but intellect 
with each composite, the results thus obtained might corre- 
spond more closely with the consensus estimates than our 
actual experimental results do. 

For this criticism to have force the Fti correlations of 
the tasks which are put as unduly hard by the consensus 
would have to be lower than the others. For example. Ft, 
for the sentence-completions should be much lower than rti 
for the arithmetic, vocabulary, or directions. This is not 
the case. We have computed the correlations of each with 
a composite made up of completions, arithmetic, vocabu- 
lary, directions, information and opposites with approxi- 
mately equal weights, using 176 of the 180 imbeciles. They 


are 

Completions with CAVDIO .90 

Arithmetic with CAVDIO .80 

Vocabulary with CAVDIO .68 

Directions with CAVDIO .92 

Information with CAVDIO .85 

Opposites with CAVDIO .92 


The range is here very restricted, all the individuals 
being within a range of 28 to 58 months of mental age, with 
a o of about 8 months. If the correlations were for all 

10 There is some spurious correlation in each of these, but this does not 
seriously damage the argument, since the amount is not large and is approxi- 
mately equal for all. 
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twenty-year-olds or for adults, they would all be very much 
higher. If we take the variability of adults as a = 2.85 
years, as computed from the army data [Memoirs, p. 391}, 
and apply the correction'^ for restricted range, we have ; 

Completions with CAVDIO .994 

Arithmetic with CAVDIO .985 

Vocabulary with CAVDIO .971 

Directions with CAVDIO .995 

Information with CAVDIO .990 

Opposites with CAVDIO .995 

In the case of 240 college graduates, the correlations of 
10-composites made up of completions, arithmetical prob- 
lems, vocabulary, and reading, respectively, with a CAVD 
summation score from 160 tasks averaged as follows: 


c 

.69 

A 

.49 

V 

.51 

D 

.56 or higher. 


Nowhere in fact do we find any inferiority of C to A, V, 
and D in closeness of correlation with i or anything ap- 
proximating to i. 

Wherever and however we estimate it, the r*! for ten of 
our sentence-completions will not be below the average of 
the correlations for ten of our A's or V's or D’s. 

We can think of no good reason why the discrepancies 
in the case of the vocabulary and reading tasks should be 
any less on the whole if we should compare them with the 
difficulties found by experiment corrected for differences 
in Tti, instead of with the mere difficulties. We may, it is 
true, hope that the experts* estimates of difficulty will dis- 
regard some of the sources of error to which the experi- 


11 Bb 


at 


See [Kelley^ *2B, p. 225.] 


VI"" 

12 Some of the D composites had fewer than 10 elements, so that the aver* 
Bge ot .56 is somewhat too low* 
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mental determinations are subject, and so deviate from the 
experimental determinations toward the difficulty for intel- 
lect uncontaminated by non-intellectual factors. But the 
particular constant errors which we have described do not 
seem alleviated in this way. 

SUMMARY 

On the whole it is certain that we cannot trust any con- 
sensus of present opinion to provide an accurate measure 
of the difficulty or of the intellectual difficulty of a single 
brief task. Psychologists do not as yet know enough about 
intellect and intellectual difficulty to avoid occasional large 
constant errors, such as the over-estimation of the difficulty 
of easy completions, or to distinguish well amongst vocabu- 
lary or reading tasks. The psychologist cannot as yet know 
from inspecting a task what fraction of intellect it will call 
into action, how high degree of intellect will be needed to 
succeed with it, and what effect non-intellectual factors will 
have upon its solution, so as to answer the question of how 
hard it will be in an actual experiment or how hard it would 
be if each person in the group used all of his intellect and 
was entirely uninfluenced by non-intellectual factors. A 
consensus of experts cannot, in the present status of psy- 
chology, either relieve us from the need of experimental 
tests of difficulty or provide an escape from our previous 
conclusion that the measurement of intellectual difficulty 
may best limit itself to composites, varied enough to utilize 
all of intellect and to equalize non-intellectual factors. 

On the other hand, the consensus estimates are in no 
sense fortuitous. The correlations of estimates with ex- 
perimental results are always positive and fairly high, even 
within the very narrow range of low-grade imbeciles, or of 
college graduates. Over a wide range the correlations will 
of course be much higher. The correspondence of opinion 
with experiment is not close enough to justify us in accept- 
ing estimates of the difficulty or of the intellectual difficulty 
of single brief tasks as always even approximately true, or 
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in leaving any result of any such estimates unchecked by 
experiment. But it is not so slight as to justify us in mak- 
ing no use of it. On the contrary, if we free it from its over- 
and under-estimation of the difficulty of certain types of 
tasks, it will give a serviceable first rough approximation 
to an order of intellectual difficulty. Even without any cor- 
rection or amendment, composites formed by taking ten of 
C, ten of A, ten of V, and ten of D, all forty of which had 

TABLE 27. 

The Differences in DiFFicuiiTY of CAVD 40-Ck)MPOsiTE Tasks by 
Experiment and by the GJonsensds of 20 Experts. 


Percent which the Stated Difference 
is of the Difference P— A 
Difference 
By the Median 
of the Four 

CoxiBensus Medians By Experiment Discrepancy 


B-A 

16.8 

8.0 

+ 8.8 

C-B 

11.0 

4.8 

+ 6.2 

D-C 

7.4 

4.1 

-t 3.3 

E-D 

5.8 

6.2 

- .4 

F-E 

2.6 

6.1 

-3.5 

G-F 

3.7 

7.7 

-4.0 

H-G 

4,2 

9,4 

~5.2 

I~H 

7.4 

16,5 

-9.1 

J-I 

10.0 

14.6 

-4.6 

K-J 

5.8 

3.5 

+ 2.3 

Ln-K 

5.3 

-.9 

+ 6.2 

M~D 

5.3 

6,0 

- .7 

N-M 

4.2 

2.1 

+ 2.1 

Q-N 

6.3 

5.7 

+ ,6 

P^O 

4.2 

5,4 

-1.2 


identical consensus estimates, would be useful composite 
tasks. We have not had time actually to make and test 
such, but we have carried out the converse procedure of 
computing the median consensus estimates for the tasks of 
our experimentally determined composites. The results 
appear in Table 27. 

The number of single tasks from the different com- 
posites which were rated by the consensus, ranged from 6 
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of P to 29 of F, being for A, B, C, D, E, F, Q, H, I, J, K, 
etc., in order 20, 13, 18, 23, 18, 29, 20, 20, 13, 15, 17, 8, 12, 16, 
13, and 6. 

The derivation of the experimental results for com- 
posites A to P is given in Chapter IX. 



CHAPTER VP 


Levels op Intellect 

We measure the level or altitude of an intellect by the 
difficulty of the intellectual tasks which it can perform suc- 
cessfully, or more exactly, by the difficulty of the tasks a 
certain defined percent of which it can perf orm successfully. 

Such tasks, to be truly intellectual, have to be com- 
posites of a number of single tasks. In the case of Intellect 
CAVD, each should represent C, A, V, and D with approxi- 
mately equal weight to each. Success with such a composite 
task may be taken to mean getting all of its single tasks 
right, or 99 percent or more of them, or any other defined 
fraction of them. For several reasons, the most useful 

1 This chapter should properly be preceded by a chapter presenting the 
facts concerniug the difficulty of the single tasks used in constructing com- 
posites with which to measure altitude or level of intellect, and concerning 
their intellectualness as measured by Ttt, or some approximation thereto, where 
that information has been obtained. 

In constructing the tests for intellectual level or altitude, we have made 
measurements of the difficulty of over three thousand tasks. The number of 
individuals concerned in one of these measurements varies from a hundred to 
over four thousand. 

These measurements are of great value quite apart from the uses which we 
have made of them. They will assist future workers in the field to extend 
and refine the selection of tests for altitude of intellect. They provide a sub- 
stantial beginning for the construction of tests of mental growth in its later 
and highe* stages, including alternative forms. They provide material for 
many scientific studies, for example, of judgments of intellect, of the organi- 
zation of ntellect, of the nature of intellectual difficulty. They may be used 
in many ways in the practical work of examining for intellect. The publication 
of such an inventory of intellectual tasks with a rough measurement of the 
difficulty o*' each will encourage others to add to it, so that after some years 
we shall have a standard source of supply of intellectual tasks of any kind, 
at any desh'ed level of difficulty. 

The expeise of ordinary publication is, however, prohibitive. Consequently, 
we have prejiared a hundred sets in the form of volumes of mimeographed 
sheets. These will be sold at cost by the Bureau of Publications of Teachers 
College, Coluiibia TJnirersify. 
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meaning to take is getting 50 percent or more of the single 
tasks right. We shall nse this meaning, unless some other 
is specified. 

COMPOSITE TASKS 

The sub-series presented in Chapter III are samples of 
the composite tasks which we have constructed; and addi- 
tional ones are shown below. 

Lbveli I 

Write words on the dotted lines so as to make the whole 
sentence true and sensible. Write one word on each inch 
of dots. 

1. Hot weather comes in the and 

weather the winter. 


2. The first after June is 

3. Children are rude not easily 


win friends. 

4. The dog a useful because 

his intelligence and faithfulness. 

5. The rose is a favorite because of 

fragrance and 

6. The poor little has nothing 

to ; he is hungry. 

7. He will come he is not ill. 

8. Not persons are eager to work hard. 

9. Divisor times quotient will dividend, if 

the is done correctly. 

10. you look, will see flowers. 

Write the answers to these problems. Use the empty space 
to figure on. 

11. What will 4 eight-cent stamps and 1 three-ceiit stamp 

cost? 

12. How many inches are there in 2 feet and 7 inches? 

13. How many stamps are there in a sheet 8 stamps wide 

and 5 stamps long? 
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14. What does a pound of candy cost when you pay 10 

cents for a quarter of a pound? 

15. How much longer is 100 minutes than an hour? 

16. 32 plus what number equals 36 f 

17. How much more is 7 X'O than 2 X 20? 

18. How long is it from seven o’clock in the morning to two 

o’clock in the afternoon? 

19. The sum of two numbers is 40. One of the numbers is 

14. What is the other number? 

20. What number added to 16 gives a number 4 less than 

27? 


Look at the first word in line 1. Find the other word in 
the line which means the same or most nearly the same. 
Write its number on the line at the right side of the page. 
Do the same in lines 2, 3, 4, etc. Lines A, B, C, and D show 
the way to do it. Do all the lines you can. Write only one 
number for each line.^ 


beast 1 afraid 2 words 3 large 4 animal 5 bird 

’iaby 1 cradle 2 mother 3 little child 4 youth 6 girl 

raise 1 lift up 2 drag 3 sun 4 bread 5 deluge 

blind 1 man 2 cannot see 3 game 4 unhappy 5 eyes 


confess 

backward 

advertise 

combat 

blond 

broaden 

chubby 

concern 

cargo 

clutch 


1 agree 2 mend 3 deny 4 admit 5 mingle 
1 downwards 2 after 3 toward the rear 4 defense 5 arrears 
1 detain 2 explore 3 give notice of 4 adverse 5 newspaper 
1 fight 2 dismay 3 club 4 expedition 5 comb 
1 polite 2 dishonest 3 dauntless 4 coy 5 fair 
1 efface 2 make level 3 elapse 4 embroider 5 widen 
1 indolent 2 obstinate 3 irritable 4 plump 5 muscular 
1 see clearly 2 engage 3 furnish 4 disturb 5 have to do with 
1 load 2 small boat 3 hem 4 draught 5 vehicle 
1 exploit 2 nest 3 hit 4 grasp 5 cane 


Read this and then write the answers. Read it again if you 
need to. 

Then, upon one knee uprising, 

Hiawatha aimed an arrow; 

Scarce a twig moved with his motion, 


s If it has not been given previously, practice or supervision should he given 
to insure that the individual tested understands these directions. 



162 


THE MEASXJREME3?T OF IKTEULIGBNCB 


Scarce a leaf was stirred or rustled ; 

But the wary roebuck started, 

Stamped with all his hoofs together. 

Listened mth one foot uplifted, 

Leaped as if to meet the arrow ; 

Ah! the singing, fatal arrow; 

Like a wasp it buzzed and stung him! 

31. What was Hiawatha trying to kill? 

32. What word is used to describe the roebuck ? 

33. What is the arrow said to resemble? 

Read this and then write the answers. Read it again if you 
need to. 

There is an old saying, ‘‘As harmless as a fly*'; and 
until recently the fly has been regarded only as an unpleas- 
ant but harmless nuisance. Had our forefathers known as 
much about flies as we now know, they might have made the • 
proverb, “As dangerous as the fly.” His origin and his 
habits are of the worst sort. He is, in short, a disgusting 
and dangerous pest. 

The scientists have told us also how to keep clear of the 
flies. Houses and grounds should be kept free of decaying 
organic matter, and stables should be screened so as to cut 
them off from their breeding places. Our houses should be 
carefully screened and food kept free from their dangerous 
feet and mouths. Fly paper and fly traps can be bought 
everyw^here. Your teacher, also, can probably tell you 
other means of protection. But don’t forget that the 
“harmless fly” of the proverb is the dangerous fly of fact. 

34. Did our great grandparents know as much about flies 

as men do now ? 

Should flies be prevented from reaching their breeding- 

places ? 

Is it desirable for a girl to be so gentle that she cannot 
bear to kill a fly? 

[All three answers must be right in 34.] 

There are only four single D Tasks in level I, so each receives a credit 
of eh 
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XjEVEL J * 

Write words on the dotted lines so as to maJce the whole sen- 
tence true and sensible. Write one word on each inch of 
dots. 

1. The way to is by airplane. 

2. There is no on earth ; cannot 

bear misfortune. 

3. Two pounds of silver are more than two 

pounds of iron. 

4. He is cheerful will make friends. 

5. A body of entirely surrounded by 

is called an 

6. The I think about it, the per- 

plexed I am. 

7. It strength to a heavy weight. 

8. When lines are perpendicular to each 

other, they form a right 

9. One times one half equals one fourth. 

10. The of five and ten is fifteen. 

Write the answers to these problems. Use the blank sheets 
to figure on. 

11. "What number minus 7 equals 23 ? 

12. What nmnber minus 16 equals 20? 

13. 12 is i of ... . 

14. If a present costing $9.45 is to be paid for by 27 men 
\ contributing equal amounts, what is one man’s 

share ? 

15. Dick started from his house, walked two miles north, 

then two miles west, then two miles south. How far 
away from his house was he then? 

16. A mstn bought land for $400. He sold it for $445, gain- 

ing $15 an acre. How many acres were there ? 

17. 12isf . 

18. Counting that 100 lb. will last 15 men for a week, how 
. much will be required to last 30 men for 3 weeks? 

19. A girl had 20 quarters, 16 dimes, 12 nickels and 8 pen- 

nies. She made four piles. Pile A, Pile B, Pile C and 
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Pile D. In Pile A she put half the quarters, one 
fourth of the dimes, one third of the nickels, and all 
of the pennies. How much money had she in Pile At 
20. At the rate of $2.25 per week how long will it take to 
save $90.00? 


Directions and samples as on page 161 . 


21. awe 

22. aged 

23. arrive 

24. blunt 

25. accustom 

26. bade 

27. bog 

28. cascade 

29. bray 

80. disembark 


1 lamb 2 fear 3 tool 4 mound 5 opera 
1 years 2 active 3 old 4 merciful 5 punctual 
1 answer 2 rival 3 enter 4 force 5 come 
1 dull 2 drowsy 3 deaf 4 doubtful 5 ugly 
1 disappoint 2 customary 3 encounter 4 get used .. 5 business 
1 gaze 2 a tool 3 fetched 4 wait 5 ordered 
1 ebb 2 disorder 3 swamp 4 held 5 difficulty 
1 hat 2 waterfall 3 firmament 4 disaster 5 box 
1 cry of an ass 2 bowl 3 cry of an ox 4 frustrate 5 raven’s cry 
1 unearth 2 ashore 3 dislodge 4 disparage 5 strip 


In each set of sentences, check the two which mean me 
nearly the same as the sentence printed in heavy type. 

••y3i. I weigh the man, not his title. — (Wycherley.) 

’Tis not the king’s stamp can make the meta 

better. 

Fine feathers make fine birds. 

Titles are the marks of honest men and wise. 

„_The rank is but the guinea stamp, the man’s the 

gold, for a’ that. 

. 32. Anyone can hold the helm when the sea is calm. 

Sail when the wind blows. 

-Untempted virtue is easily retained. 

The pilot cannot mitigate the billows or calm 

the winds. 

A.n unassaulted castle is easily held. 

33. In the presence of the greater malady, the lesser is 
forgot. 

We see not the candle if the moon be shining. 

A.n ounce of prevention is worth a pound of cure. 

The greater glory dims the less. 

There are some remedies worse than the disease. 
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34. What is failure? It is only a spur to the one who 

receives it in the right spirit. 

Every rebuff is a stepping stone to higher 

things. 

To reach the pOrt of heaven we must sail, and 

not drift, nor lie at anchor. 

Failure makes the spirit within stir to go in once 

more and fight. 

Not failure, but low aim is crime. 

The paragraph for task 35 is the Hiawatha paragraph 
on page 161, 

35. What two words are used to tell the noise the arrow 

made? 

The paragraph for tasks 36, 37, and 38 is the 
on page 16.2. 

36. Where does the paragraph say the fly is born? 


37. Who or what informs us how to avoid the dangerous 
pest described in the paragraph printed above ? 


38. Name three devices which protect us from the disgust- 

ing pest. 

EVEEY HOME NEEDS A GARDEN 

A MAGAZINE published to promote real gardening. 
Most people do not think much about their gardens at this 
time of the year, but if more people did, there would be 
more good gardens. If you live in the city where space is 
at a premium, we provide pleasure for you by suggesting 
how to grow flowers indoors. If you live in the country 
and have a garden and do not experience the satisfaction 
of seeing things grow as a result of your own efforts — ^then 
' 70 U need the X.Y.Z. magazine. 

39. What is recommended for persons who fail to make 

tMngs grow in their gardens ? . 
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40. Which one of these words could best be used instead of 
at a premium f Draw a line under it. 

space flowers valuable extension extensive 
cheap noble 

Level K 

Write words on the dotted lines to make the whole sentence 
true and sensible. Write one word on each inch of dots. 

1. When a man is of sight, 

also very soon out .mind. 

2. No is powerful to 

two and two be five. 

3. - 3 "ou wish me to help you — 

Latin, please me by telephone. 

4. He is genteel who does deeds. 

5. It may effort and a long but 

the result is sure. 

6. This magazine is the of a new and j)ro- 

gressive movement. 

7. Four two is more seven. 

8. No .what happens wrong is right. 

9. The source wealth in Denmark 

agriculture. 

10. In to maintain health, one 

should have nourishing. 

Write the answers to these problems. Use the blank sheets 
to figure on. 

11. A man spent two thirds of his money and 

How much had he at first? 

12. I bought 4J yards of cloth, gave the c 

ceived 20 cents as correct change, 
price of the cloth per yard? 

13. A dealer bought some mules for $800. He 

for $1,000, making $40 on each mul^ HoiW 
mules were there? 

14. How much more is the sum of 3J and 44 

of 2f and 34? 
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16» 20 = how many times 12 ? 

16. How many times must you add 1^ to 6 to have 15 as 

a result? 

17. How many times as long as 8 feet is 12 yds. ? 

18. 20 = |X 

19. 8 is li X 

20. 20 is li X 

Directions and samples the same as on page 161 . 


21. conspire 1 plot 2 breathe 3 rely 4 die 5 outrun 

22. check 1 error 2 stop 3 dash 4 rude 5 haste 

23. cherish 1 dedicate 2 happy 3 covet 4 hold dear 5 marry 

24. chirrup 1 aspen 2 joyful 3 capsize 4 chirp 5 incite 

25. accessible 1 indefatigable 2 successful 3 limpid 4 easy to reach 5 liable 

26. dingy 1 afraid 2 hostelry 3 small bell 4 midget 5 dirty 

27. edible 1 auspicious 2 eligible 3 ht to eat 4 sagEiciouB 5 able to speak 

28. confound 1 discovered 2 fulfill 3 establish 4 mix up 5 expire 

29. concur 1 agree 2 race 3 mongrel 4 pounce 5 ramble 

30. contact 1 tactful 2 hate 3 injunction 4 touch 5 


olf 


>in the space h. 

In each set of sentences, check the. 
nearly the same as the sentence 

_ j . ^1. ^ ^ tinted in heavy type 

31. Today is 

r7' tomorrows. 

Time IS an herb that cures all diseases. 

A bird in the hand is worth two in the bush. 
^ speed today is to be set back tomorrow 
like the present. 

32. Faint heart never won fair lady. 

venture; nothing g^iin. 

i$i haste we repent at leiiSlx^ 1 . 


-a**. , favoi» the brave, 

fire wiUi fire. 

-Set a thief to catch a thief. 
Knavery is the best defence aii 
Sq3v the wind, reap the whirh 
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On horror’s head horrors accumulate. 

Sorrow’s crown of sorrow is remembering hap- 
pier things. 

The paragraph for task 35 is ** Every Home Needs a 
Garden/* on page 165, 

35. What does the advertisement say would be the result 

if i^eople thought more about their gardens in the 

time of year referred to? 

The paragraph for task 36 is the paragraph on Work 
on 'page 81. 

36. According to the paragraph what even would a prisoner 

welcome? 


The paragraph for task 37 is "'The American State/* 
on page 84. 

37. In ^^resull^®®^ some of the original thirteen states 

-.ns magazine ili all the others? — — 

gressive Tr 

^ 4 ?*^ 38, 39, a'nd 40 is Dirge in 
The paragraph for tasKu 

Woods/* on page 94. 


38. To whom does ‘^we” refer? 

39. What veins the moss carpet?-.. 

40. What event of man’s career is 

fruits of the pine? 


like the falling of the 


Level L 

Write words on the dotted lines so as to -.nahe the whole 
sentence true and sensible. Write one word on each inch 

hf dots. 

1. Many new are printed every year, bnt 

some wise prefer to 

old ones. 

2. Telephone and were means of 

unknown in the seventeenth — • 

3. Much of the debris 

is valuable. 
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4. Most men themselves more 

kindly than their judge them. 

5. Power is generated , gaso- 

line, and several things. 

6. A of ease is preparation for 

achievement. 

7. Sailors fear most and snowy 

because then there is most of a collision 

between 

8. She was to fashion fine — 

from the cloth she had learned to 

9. The of the river are being constantly 

by the of the water. 

10. Any will stick to a master is 

and kind to 


In the lines below, each number is gotten in a certain 
way from the numbers coming before it. Study out what 
this way is in each line, and then wtite in the space left for 
it the number that should come next. The first two lines 


are already filled in as they should be. 



SAMPLES 

{i? 

4 

12 

6 

14 

8 

15 

10 

17 

1.8™- 

— JA™ . 

11. 




85 

79 

73 

67 



12. 




90 

81 

72 

63 


13, 




76.3 

85.3 

94.3 103.3 


14. 




64 

32 

16 

8 


15. 




240 

120 

60 

30 

i >.•' 

16. 


12 

16 

22 

26 

32 

36 

> 

— A.a, — 

17. 

7 11 

15 

16 

20 

24 

25 

29 


18. 




13 

12§ 

12^ 

12 


19. 




46 

45f 

45i 

45i 

i 

20. 

2 5 

7 

8 

11 

13 

14 

17 

— 1.^^! — 


Directions and samples as on page 161. 


21. downcast 

22. pact 

23. audible 

24. solicitor 

25. beguile 

26. dominate 

27. average 


1 thrown down 2 neutral 3 judicious 4 sad 5 broken 
1 puissance 2 remonstrance 3 agreement 4 skillet 5 pressure 
1 festive 2 easy 3 audit 4 heard 6 downy 
1 lawyer 2 chieftain 3 watchman 4 maggot 5 constable 
1 entreat 2 delight 8 dispense 4 deceive 5 foster 
1 abide 2 goad 3 threaten 4 control 5 dissuade 
1 level 2 count 3 evident 4 ordinary . 6 distinct 
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28. behave 1 act 2 own 3 keep still 4 enable 5 entitle 

29. comelj 1 ignoble 2 handsome 3 disagreeable 4 enter 5 in time 

80. cycle 1 scythe 2 cyclone 3 circle 4 ode 5 junction 

In each set of sentences, check the two which mean most 
nearly the same as the sentence printed in heavy type. 

' 31. Man’s evil manners live in brass; their virtues we 
write in water. — (Shakespeare.) 

Some rise by sin and some by virtues fall. 

The evil that men do lives after them; the good 

is oft interred with their bones. 

He lives in fame that died in virtue’s cause. 

The memory of vices lives longer than the mem- 
ory of virtues. 

32. In this world a man must either be anvil or hammer. 

To get along, a man must be a knocker. 

Man must either do or be done. 

Man cannot be neutral; he must accomplish 

something or lose out. 

Might is right. 

33. No greater grief than to remember days of joy when 

misery is at hand. — (Dante.) 

Misery loves company. 

Sorrows remembered sweeten present joy. 

To recall past pleasures is but to aggravate our 

present miseries. 

A sorrow’s crown of sorrow is remembering 

happier things. 

34. It is a consolation to the wretched to have com- 

panions in misery. 

Society in shipwreck is a comfort to all. 

Misery acquaints a man with strange bedfellows. 

Company in misery makes it light. 

^When misery is highest, help is nighest. 

35. Nothing emboldens sin so much as mercy. 

Spare the rod and spoil the child. 

Pardon one offense, and you encourage the com- 
mission of many. 
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He that has no charity merits no mercy. 

Let the punishment fit the crime. 

36. He counsels best who lives best. 

Practice what you preach. 

A. poor cask may hold good wine, 

A. good example is the best sermon. 

An ounce of prevention is worth a pound of cure. 

37. The fault is not in our stars but in ourselves, that 

we are underlings. — (Shakespeare.) 

Every man is the architect of his own fortune. 

Man is the master of his destiny. 

.We can’t read our fates from the stars. 

If we are underlings, it is not our fault. 

38. Every white will have its black, and every sweet 

its sour. 

It never rains but it pours. 

Birds of a feather flock together. 

Every cloud has a silver lining. 

You cannot pluck roses without thorns. 

The paragraph for task 39 is the ‘*Fly” on page 162. 

39 . What is meant by ^*his origin”? 

The paragraph for task 40 is Every Home Needs a 

Garden** on page 165. 

40 . What feeling is usually said to be experienced by people 

who see things grow as a result of their efforts? 


Level M 

Write words on the dotted lines sa^as to make the whole sen- 
tence true and sensible. Writik one word on each inch of 
dots. 

1 . 

2 . 

3 . 
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4. More were killed, more houses , 

more money during the Great , 

than during any equal number of in 

history. 

5. In the time squirrels store 

for food in the when the 

is such that they cannot for things to 

eat. 

6. Columbus America, but it was 

for another Italian, 

7. A boy on a farm things about 

animals and which a city 

usually does not 

8. The wind the streets of 

every flake of 

9. The benefit to the from the 

of science is incalculable. 

10. The old days are often with 

the jiresent. 


Tasks 14 and 15 must both be right to secure credit. 
There are thus only 8 tasks; and each counts as 1^. 


A 3 for 5c. 

B 3 for 10c. 

C 3 for 25c. 

D 48c. per lb. 


E 6 for 5c. 

P 80c. per peck. 
G 40c. per lb. 

H 50c. per lb. 

I 8 for $1.00. 


J 4 for 25c. 
K 2^e. each. 

L l^c. per lb. 
M 4^c. per lb. 


A, B, C, B, etc., are articles costing as shown dbot^e. lA 
means 1 of A, 2 A means 2 of A, 3 A means 3 of A, etc. Sup- 
ply the missing numbers in lines 11 to 15 as shown in lines 
I, II, and III. Use the empty parts of the page to figure on. 


11 . 

12 . 

13. 


I. 3 A cost ^ as much as 3 B. 

11. 2| lb. G cost just as much as 8 I. 

III. 1 lb. fT costs 2 times as much as 4 J. 

\ r 

^ one offen8€““ ^ 

1 lb. H. 


i«n one offenscT*" 
x£tj!i38ion of many. 
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14. 2 lbs. M “ ‘‘ 1 lb. L. 

15. 1 lb. L “ ‘‘ 1 lb. M. 


Write the numbers and signs in each line below in the 
proper order, so that they make a true equation as shown 
in the three sample lines. Use the bottom of the page to 
figure on if you need to. 

(3 3 6 = + 3 + 3 = 6 

Sample lines/! 78 20 = + X 7Xd = 20 + 8 

(2 3 3 7 18 = + — X( ) 7 + 2 = 18--{3X3) 


16. 

2 

2 

3 

5 

15 = — 

- X 

17. 

1 

1 

4 

4 

16 = — 

- X 

18. 

2 

5 

6 

7 

10 = + 

+ — 

19. 

1 

4 

8 

15 

20 = + 




20. Counting that 25 dozen sheets of paper are worth ten 
cents, how many sheets of j)aper are wortli a fifth 
of a cent ? 

Directions and samples the same as on page 161. 


21. action 

22. avarice 

23. bearing 

24. allusion 

25. dynasty 

26. habitat 

27. adversity 

28. caprice 

29. ignominious 

30. chastity 


1 play 2 deed 3 mention 4 opinion .*) crime 

1 ordinary 2 various 3 empress 4 frailty 5 greed 

1 a largo ring 2 behavior 3 cub 4 commendation 5 destination 

1 aria 2 illusion 3 eulogy 4 dream 5 reference 

1 davenport 2 very unpleasant 3 framework 4 ruling family 5 engine 

1 dweller 2 bodice 3 prodigality 4 habit 5 home 

1 ill fortune 2 dialogue 3 advertisement 4 dislike 5 distemper 

1 value 2 a star 3 grimace 4 whim 5 inducement 

1 seductive 2 not guilty 3 incontestable 4 ignorant 5 shameful 

1 dissension 2 pursuit 3 eminence 4 jiurity 5 punishment 


In each set of sentences, check the two which mean most 
nearly the same as the sentence printed in heavy type. 

31. What a man has, so much is he sure of. 

There’s many a slip ’twixt the cup and the lip. 

Tie who hesitates is lost. 


Look before you leap. 

A bird in the hand is worth two in the bush. 

32. Tho the knowledge they (the ancients) have left 
us be worth our study, yet they exhausted not 
all its treasures; they left a great deal for the 
industry and sagacity of after ages. — (Locke.) 

Worth is wholly dependent in long use. 

Build the present on a knowledge of the past. 
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.Do not neglect the present in admiration of the 

past. 

There is nothing new under the snn. 

33. Cowards die many times before their death. — 

( Shakespeare. ) 

Fortune favors the brave. 

Discretion is the better part of valor. 

The valiant never taste of death but once. 

They suffer more who fear than they who die. 

34. Some books are to be tasted, others to be swal- 

lowed, and some few to be chewed and digested. 
— (Bacon.) 

Beading is profitable to every one. 

.One should read only parts of some books, while 

others should be carefully studied. 

Only a few books repay one for painstaking 

effort. 

People's tastes differ in books. 

35. Write it on your heart that every day is the best 

day of the year. — (Emerson.) 

There is no time like the present. 

Never do today what you can put off until to- 
morrow. 

Anticipation is better than realization. 

A. common delusion is that the present hour is 

not the critical, decisive hour. 

36. Our virtues disappear when put in competition 

with our interests. — (La Rochefoucauld.) 

A dog with a bone knows no friend. 

My te(‘th are nearer than my kindred. 

.Virtue is its own reward. 

A good friend is my nearest relation. 

37. If men wish to be held in esteem, they must asso- 

ciate with those only who are estimable. — (La 
Bruyere.) 

What a man does shows what he is. 

You cannot always judge a man by his sur- 
roundings. 
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T Tfi who comes from the kitchen smells of its 

smoke. 

Jf you always live with those who are lame, you 

will yourself learn to limp. 

38. We too often forget that not only is there a soul of 

goodness in things evil, but very generally also a 
soul of truth in things erroneous. — (Spencer.) 

Falsity frequently has a nucleus of reality. 

Beliefs that are shown to be untrue may, never- 
theless, be based on some element of truth. 

Benevolence sometimes has evil consequences. 

Evil is commonly due to error. 

39. They build too low who build beneath the stars. 

Not failure, but low aim is crime. 

ITitch your wagon to a star. 

lie that strives to touch a star often stumbles at 

a straw. 

.Wouldst thou reach stars because they shine on 

thee? 

The paragraph for tash 40 is ** Every Home Needs a Gar- 
den,” on page 165, 

40. Copy the four words which most fully state the pur- 

pose of the X. Y. Z. magazine. 

The Construction of Composite Tasks 
With the knowledge gained in the course of our investi- 
gations, we could now construct composite tasks for use in 
measuring altitude or intellect which would be much supe- 
rior to these. But these will serve reasonably well. 

If we had begun our work with the knowledge which we 
now have, we should also have proceeded somewhat differ- 
ently in their construction. The procedures which we did 
use will consequently be reported here only very briefly. 
We shall preface them by a description of a more efficient 
and economical method of construction of such composite 
tasks, which we recommend for the future. 



176 


THE MEASUBEMEKT OF 1I!TTELX.IG£NCE 


It is as follows: Select the special abilities which to- 
gether constitute the sort of intellect (call it intellect 
abc . . . n) for which composite tasks are to be constructed. 
Select a sufficient number of single tasks to provide one 
hundred for each special ability that is included at each 
twentieth of the total range of intellect abc , . . n from the 
lowest thousandth of human adults to the highest thou- 
sandth (or the proper segment of such a collection, if the 
tasks are to cover only a part of this range). In this selec- 
tion you trust your own knowledge and judgment. Have 
twenty or more competent judges rank these tasks for intel- 
lectual difficulty for the group whose intellect abc . . . n 
you plan to measure by the tasks. Let them use as fine a 
scale as is convenient up to two hundred compartments, 
and require the use of approximately the same number of 
compartments by each judge (say, 150 to 200, or 75 to 100, 
or 60 to 75, or 45 to 60, or 32 to 45, or 25 to 32, or 18 to 25). 
Express the results of this consensus by simple summing. 
Arrange the single tasks in order of difficulty as estimated 
by the consensus, and in series representing each the same 
special ability (unless some better way is found to insure 
that persons to be tested understand the general nature of 
the tasks, and do not fail because of misunderstanding 
directions). 

Test with a cross-section of these tasks from fifteen hun- 
dred to twenty -five hundred individuals, taking about two 
hundred from each of ten groups selected to represent dif- 
ferent altitudes of intellect abc . . . n, such as, college 
graduates, pupils in grade 1 2, pupils in grade 9, . . . adults 
of mental age 4. Let the tasks used always begin at a point 
where 95% of the group of two hu?idred can succeed with at 
least four out of five of the tasks. Be sure that each indi- 
vidual has sufficient time. It will be found most convenient 
to have each individual in the group attempt all of the tasks 
used with that group. 

Enter the score as e, x, or — (correct, wrong or omitted) 
for each individual in each group for each task. Find the 
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percent of successes for each task in each group. Make 
up composites containing 2 tasks of a, 2 tasks of b, 2 tasks 
of c ... 2 tasks of n, putting in one such composite tasks 
most nearly alike in difficulty. Call such a composite a 2n- 
composite. Find the percent of successes for each 2n-com- 
posite in each group which was tested by all its tasks. Plot 
the successes and failures’ in each 2n-composite in at least 
one group* against the total score (number of tasks cor- 
rect), and compute the overlapping of the failures past the 
median of the successes in that group. Compute the bi- 
serial r. 

Combine the 2n-composites into 4n or 6n or 8n or lOn or 
12n composites, using 2n composites which are neighbors 
in difficulty, and making each composite large enough so 
that its rti will be at least .90 for a grade population or 
other group of approximately the variability of a grade 
population. How large composites will be needed can be 
judged from the size of rti for the 2n composites, the self- 
correlations of the 2n composites, and the self-correlation 
of the measure of i. This last® should be approximately 
1 . 00 . 

The resulting composites should be nearly or quite as 
satisfactory for measuring intellect abc ... n as the 40- 
coinposites described in tliis chapter are for measuring In- 


3 A success in a 2n -composite is a case which has n or more ri|i^ht. A 
failure is a case which has fewer than n right. 

4 Use the group which most nearly ai)proximates 50% of successes with 
the 2n-compoBites. 

® Uct Tti^ = the average r from the 2n composites. 

=the average self -correlation for a 2n composite. 

** r,^ =-the self-correlation of the measure of i. 

“ Ttx =the average self -correlation of a composite necessary to produce 
an Tti of .90. 


Then .90 = - 




and n, the number of 2n composites necessary to produce a self -correlation of 
can be computed from 
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tellect CAVD. The Th for any one of them should be very 
close to 1.00 for all adults, or for any group of the same 
chronological age. All the tasks in any one of them will be 
enough alike in difficulty to seem neither much too easy nor 
much too hard to those for whom the composite as a whole 
is suitable. 

The same procedure may be followed in constructing 
levels for any ability •which has what we have termed alti- 
tude,^’ that is, which has to master tasks varying in diffi- 
culty. The difficulty may be in -words that are harder to 
spell, that is, require a higher altitude of spelling ability 
for success ; or in temptations to dishonesty that are harder 
to resist, that is, require a higher altitude of honesty to 
pass ; or in hundreds of other sorts of tasks. But where^ver 
the concepts of difficulty and altitude are applicable, this 
method of constructing measuring instruments is appli- 
cable. 

At the outset of our studies, we lacked the knowled. e 
of how often and how far a consensus of expert judgt 
could be trusted in its estimates of intellectual difficulty '' 
and the knowledge of how many single elements are needeu 
to give a reliable measure of intellectual difficulty, and the 
knowledge of the essential impossibility of measuryag the 
intellectual difficulty of any single small task. So we did 
not proceed in the way outlined above, but began with single 
small tasks, estimated their difficulty by the percent of vari- 
ous groups which succeeded with each, combined these into 
composites by special abilities, that is, into sets of ten or 
twenty completions of approximately equal difficulty; sets 
of ten or twenty arithmetical problems of approximately 
equal difficulty, and so on. The 40 element composites were 
made by putting together a 10 completion composite, a 10 
arithmetic composite, a 10 word knowledge composite, and 
a 10 sentence-comprehension composite, which were, as 
composite tasks, as nearly equal in difficulty as could be 
found in our material. 

This method does have the advantage that we b 
means of conveniently measuring the difficulty of tas’ 
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these four abilities separately, and have made many such 
measurements of value (these are reported in Chapter 
VIII). The disadvantages are that our composite tasks do 
not represent as narrow segments or slices of difficulty as 
they might have done; kre not spaced apart as evenly as 
they might have been, and required much more labor in 
their construction than would have been the case by the 
other method. 

We shall describe briefly the derivation of the word- 
knowledge composites of ten single tasks as a sample to 
show the nature and validity of the selection and the extent 
of the experimentation involved. In the case of the others 
we shall simply present the evidence that the elements of 
each composite of ten (occasionally fewer), do belong fairly 
in that rather than in an easier or harder composite. We 
shall then even more briefly relate samples of the evidence 
by which these composites of ten were put into composites 
of forty. Finally we shall state the facts concerning the 
value of the composites of forty as intellectual tasks the 
difficulty of which we shall later measure. 

10-CoMPOsiTEs IN Word Knowledge or V 

Consider the tasks shown below. Each ‘Level’ or 10- 
Composite is, by our definition of difficulty, harder than 
the preceding for such a group as persons twelve to twenty 
years old or older who have lived in the United States five 
years or more, since a smaller percentage of them will get 
five or more of the ten elements right. The difficulty is 
‘intellectual’ to the extent that within any sub-group of 
equal age the greater intellects will show higher percents 
correct than the smaller intellects in the case of any word. 

It may seem far-fetched and forced and an unhappy 
consequence of our definitions to argue thus that it requires 
more intellect to know such words as cloistered, madrigal 
and ignominious, than to know such words as confess, ad- 
vertise and combat. A dull person, it may be said, could 
learn the former as well as the latter ; and it is a matter of 
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range rather than level that he does not. There is much 
force in this criticism, and we chose the case of Word 
Knowledge as one illustration of the measurement of dif- 
ficulty, in order to state the answer to the criticism. 

Word Knowledge is representative of many tasks of an 
informational character where many of the harder tasks 
might have been in the repertory of the dull so far as the 
essential difficulty of mastering them is concerned, but sim- 
ply are not as a matter of observed fact. They are not 
there because the greater intellect can learn more per unit 
of time and has learned more at equal age ; range is posi- 
tively correlated with level. Also there is, for any locality 
and epoch, a certain rough order of acquisition, whereby 
people usually do not progress to learn certain things until 
they have learned certain other things. The former are 
then ‘harder’ by our definition although, if customs had 
been reversed, they might have been easier. 


Look at the first word in line 1. Find the other word in 
the line which means the same or most nearly the same. 
Write its number on the line at the right side of the page. 
Do the same in lines 2j 5, 4, etc. Lines A, B, C, and D show 
the way to do it. Do all the lines you can. Write only one 
number for each line. 


A. beast 

B. babj 
raise 

B. blind 


1 afraid 2 words 3 large 4 animal 5 bird 
1 cradle 2 mother 3 little child 4 youth 5 girl 
1 lift up 2 drag 3 sun 4 bread 5 deluge 
1 man 2 cannot see 3 game 4 imhappy 5 eyes 


Begin: 

1. await 

2. beautify 

3. bug 

4. arrange 

5. different 

6. cotton 

7. blacken 

8. ablaze 

9. avenue 
10. bench 


Level VI 

1 pace 2 slow 3 wait for 4 tired 5 quit 
1 make beautiful 2 intrude 3 exaggerate 4 insure 5 blessed 
1 insect 2 a vehicle 3 ffber 4 abuse 5 din 
1 put in order 2 hasten 3 distance 4 frighten 5 charge 
1 not the same 2 quarrelsome 3 better 4 complete 5 not here 
1 cloth 2 small bed 3 hut 4 flour 5 herd 
1 a fern 2 interpose 3 impel 4 make black slack 
1 ostensible 2 on fire 3 slightly 4 loaf about 6 urbane 
1 justice 2 arrival 3 street 4 jury 5 library 
1 tool 2 pull ashore 3 opinion 4 seat 5 pond 
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11. confess 

12. backward 

13. advertise 

14. combat 

15. blond 

16. broaden 

17. chubby 

18. concern 

19. cargo 

20. clutch 


21. awe 

22. aged 

23. arrive 

24. blunt 


25. accustom 
20. bade 
27i bog 


28 ^ 

29 , 

30. 


cascade 

bray 

disembark 


31. conspire 

32. cheek 

33. cherish 

34. chirrup 

35. accessible 

36. dingy 
W. edible 
K. confound 
B concur 
^ contact 


lowncast 

pact 

audible 

solicitor 

(beguile 

|loxninate 

^average 

fbehave 

jkcomely 

^jyde 


Level V2 

1 agree 2 mend 3 deny 4 admit 5 mingle 
1 downwards 2 after 3 toward the rear 4 defense 5 arrears 
1 detain 2 explore 3 give notice of 4 adverse 5 newspaper 
1 £ght 2 dismay 3 club 4 expedition 5 comb 
1 polite 2 dishonest 3 dauntless 4 coy 5 fair 
1 efface 2 make level 3 elapse 4 embroider 5 widen 
1 indolent 2 obstinate 3 irritable 4 plump 5 muscular 
1 see clearly 2 engage 3 furnish 4 disturb 5 have to do with 
1 load 2 small boat 3 hem 4 draught 5 vehicle 
1 exploit 2 nest 3 flit 4 grasp 5 cane 

Level V3 

1 lamb 2 fear 3 tool 4 mound 5 opera 
1 years 2 active 3 old 4 merciful 5 punctual 
1 answer 2 rival 3 enter 4 force 5 come 
1 dull 2 drowsy 3 deaf 4 doubtful 5 ugly 
1 disappoint 2 customary 3 encounter 4 get used 5 business 
1 gaze 2 a tool 3 fetched 4 wait 5 ordered 
1 ebb 2 disorder 3 swamp 4 field 5 difficulty 

1 hat 2 waterfall 3 firmament 4 disaster 5 box 

1 cry of an ass 2 bowl 3 cry of an ox 4 frustrate 5 raven 's cry 
1 unearth 2 go ashore 3 dislodge 4 disparage 5 strip 

Level V4 

1 plot 2 breathe 3 rely 4 die 5 outrun 
1 error 2 stop B flash 4 rude 5 haste 
1 dedicate 2 happy 3 covet 4 hold dear 5 marry 
1 aspen 2 joyful 3 capsize 4 chirp 5 incite 

1 indefatigable 2 successful 3 limpid 4 easy to reach 5 liable 

1 afraid 2 hostelry 3 small boll 4 midget 5 dirty 
1 auspicious 2 eligible 3 fit to eat 4 sagacious 5 able to speak 
1 discovered 2 fulfill 3 establish 4 mix up 5 expire 
1 agree 2 race 3 mongrel 4 pounce 5 ramble 
1 tactful 2 hate 3 injunction 4 touch 5 oversight 

Level V5 

1 thrown dovrn 2 neutral 3 judicious 4 sad 5 broken 
1 puissance 2 remonstrance 3 agreement 4 skillet 5 pressure 
1 festive 2 easy 3 audit 4 heard 5 downy 
1 lawyer 2 chieftain 3 watchman 4 maggot 5 constable 
1 entreat 2 delight 3 dispense 4 deceive 5 foster 
1 abide 2 goad 3 threaten 4 control 5 dissuade 
1 level 2 count 3 evident 4 ordinary. 5 distinct 
1 act 2 own 3 keep still 4 enable 5 entitle 
1 ignoble 2 handsome 3 disagreeable 4 enter 5 in time 
1 scythe 2 cyclone 3 circle . 4 ode . 5 junction 
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51. action 

52. avarice 

53. bearing 

54. allusion 

55. djnastj 

56. habitat 

57. adversity 

58. caprice 

59. ignominious 

60. chastity 


Level V6 

1 play 2 deed 3 mention 4 opinion 5 crime ^ ^ 

1 ordinary 2 various 3 empress 4 frailty 5 greed 
1 a large ring 2 behavior 3 cub 4 commendation 5 destination 
1 aria 2 illusion 3 eulogy 4 dream 5 reference 
1 davenport 2 very unpleasant 3 framework 4 ruling family... 

5 engine 

1 dweller 2 bodice 3 prodigality 4 habit 5 home 
1 ill fortune 2 dialogue 3 advertisement 4 dislike 5 distemper 
1 value 2 a star 3 grimace 4 whim 5 inducement 
1 seductive 2 not guilty 3 incontestable 4 ignorant 5 shameful 
1 dissension 2 pursuit 3 eminence 4 purity 5 punishment 


61. gainsay 

62. eclogue 

63. cloistered 

64. reciprocal 

65. accolade 

66. benighted 

67. madrigal 

68. pinnace 

69. broach 

70. nectarine 


Level V7 

1 persuade 2 beshrew 3 deny 4 profit 5 imprint 
1 obituary 2 a poem 3 carousal 4 epigram 5 portrait 
1 miniature 2 bunched B arched 4 malady 5 secluded 
1 saturnine 2 mutual 3 receptive 4 morose 5 careless 
1 salutation 2 anchovy 3 procession 4 bivouac 5 acolyte 
1 fraudulent 2 weary 3 insuperable 4 ignorant 5 venal 
1 song 2 mountebank 3 lunatic 4 ribald 5 sycophant 
1 a boat 2 doublet 3 pinnacle 4 hold fast 5 forfeiture 
1 dodge 2 clasp 3 open 4 top 5 edify 
1 bouillon 2 a fruit 3 a jewel 4 a drink 5 diurnal 


Intellectual tasks range in this respect between two ex- 
tremes. At one extreme the tasks arc, in and of themselves, 
almost or quite impossible for the dull person regardless 
of which things the world tries to teach him. At the other 
the tasks are such as he can master nearly or quite as easily 
as he can master any intellectual tasks, the question being 
rather how many a dull person can master at a given age or 
with a given set of opportunities. For example, two of our 
very hard word tasks are: % 


reciprocal 

morose careless 

nectarine bouillon 

drink diurnal 


-a fruit- 


-a jeweL 


.a 


A person twenty years old with a mental age of four not 
only would not know the meaning of reciprocal, but also 
probably never could he taught it. The idea involves think* 
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_by aspects and in relationships in a way that 
[ond his degree of intellect. He would not, 
stances, know nectarine j hut with proper 
^naow nectarine instead of some word, say 
es kilow. 

best to measure level or ** altitude'' 
it lie toward the former extreme; 
iS at so^w e may, in general, expect 
IT of-'lflmfe spent from using such. 
They are likely to involve more of intellect, and lo be less 
adulterated by other influences than intellect, and to be 
more representative of level and less of width or range.* 
However, the standard tests used for measuring intelli- 
gence contain tasks that range far toward the other ex- 
treme, and it is obviously desirable to measure the diffi- 
culty of these tasks and ascertain how much of it is due to 
intellect pure and simple, and how much of it is due to other 
factors. 



Word Knowledge is a specially suitable case for study, 
because it has been approved by Terman as one of the very 
best single measures of intellect, and is involved to some 
degree in many of our better tests, such as oral and printed 
directions, paragraph reading or comprehension, sentence 
completion, opposites, and other tests of relations pre- 
sented in words. 

We began with four hundred words chosen originally 
to make an instrument for measuring word knowledge with- 
out regard to the merits or demerits of any one of them as 
a measure of intellect. 

The selection amongst these was made solely on grounds 
of the percentages right in certain groups, the end sought 
being to have for any one level word-tasks which were ap- 
proximately equally hard in the sense of being done cor- 
rectly by approximately equal percents of the group; and 


^ These matters will be treated in connectioit with ztew experimental da tSi 
to be presented in Chapter XV. We shall there see that the theoretieal and 
practical advantages are much less than has been supposed. 

14 
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to have, at the next higher level, words which were done by 
fewer of the group. ' 

The procedure vras as follows : 400 words, ranging CT&m 
very common words to words far outside the first ten thou- 
sand as listed in the Thorndike Teachers Word Book, were 
used in the case of 278 pupils in grade nine. On the basis 
of the percents correct, 110 of the tasks were chosen, 


10 

done correctly by 270 

or 

277 

or 

99.3 

to 99.6% 

of the pupils 

10 

£t 

4 4 

4 4 

271 

to 

273 

or 

97.1 

to 

97.8% 

it 

4 4 

44 

15 

4 € 

4 4 

4 4 

257 

to 

261 

or 

92.4 

to 

93.9% 

4 4 

4 4 

4 4 

15 

4 4 

4 4 

4 4 

228 

to 

236 

or 

82.1 

to 

84.9% 

4 4 

€4 

4 4 

15 

4 4 

4 4 

44 

185 

to 

194 

or 

66.6 

to 

69.8% 

it 

4 4 

4 4 

15 

4 4 

4 4 

4 4 

134 

to 

143 

or 

48.2 

to 

51 . 5 % 

44 

4 4 

44 

15 

4 4 

4 4 

4 4 

79 

to 

90 

or 

28.4 

to 

32.4% 

4 4 

4 4 

4 4 

15 

1 4 

4 4 

4 4 

37 

to 

51 

or 

13.3 

to 

18.3% 

4 4 

4 4 

4 4 


These 110 tasks were experimented with in the case of 
430 pupils in grades 11 or 12, 500 pupils in grades 9 or 10, 
250 pupils in grade 8^, and 514 pupils in grade 6, and 
smaller groups of college students. 

From them were chosen the seven ‘Levels* of ten tasks 
each shown above. Levels 1 and 2 were constructed chiefly 
on the basis of the results with the 514 pupils of grade 6. 
Levels 3, 4, and 5 were constructed chiefly on the basis of 
the results with pupils of grades 9 to 12. Levels 6 and 7 
were constructed chiefly on the basis of the results with 
pupils ill grades 11, 12, 13, and 17. The tasks within any 
one level vary in difficulty somewhat widely and it is pos- 
sible that results from as many thousands as we have hun- 
dreds might show some tasks in adjacent levels which actu- 
ally should be transposed. 

Greater equality within and distinctness between levels 
could have been attained by reducing the number from ten 
to eight or fewer, but this did not, on the whole, seem de- 
sirable. The order of difficulty of these tasks varies so 
much from group to group, and so enormously from one 
individual to another that, at levels where a person gete 
from 20% to 80% right, the percent which an individual has 
correct from one of our sets of ten is probably a more re- 
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liable measure of the percent which he would have correct 
from a hundred tasks each of exactly the same difficulty as 
the median task of the ten than is the percent which he 
would have correct of the middle eight of the ten. Diffi- 
culty is taken in the above to be difficulty for the sort of 
persons who get about half right at the level in question. 

The essential facts concerning the percentages correct 
for each of the 110 tasks are shown in Table 28. 

TABLE 28 

Percents Correct for Each SinoijE Word of Seven 10-Word Composite 
Tasks in Each of Various Groups of Individuals 



Grade 

6a 

8% 

9 

9 + 10 

11 + 12 

12 

12 


City 

Number of 

N.T. 

N. Y. 

Mix. 

K 

K 


K, 


Individuals 

514 

250 

278 

500 

430 

200 

200 

1 

await 

94.9 

90.0 

99 6 

92.8 

97.9 

95.5 

96.5 

3 

beautify 

94.3 

93.2 

99.3 

94.6 

94.2 

97 5 

94.0 

6 

bug 

94.6 

94.4 

99.6 

97.8 

99.5 

99.5 

99.0 

7 

arrange 

96 0 

95.6 

99.3 

97.2 

99.3 

98.5 

98.5 

9 

different 

94.5 

93.0 

99.3 

97.4 

300.0 

99.5 

96.5 

10 

cotton 

93 4 

93.2 

99.6 

96.4 

98.8 

96.0 

96.5 

12 

blacken 

95.2 

94.4 

97 5 

98.0 

99 3 

98 5 

98.0 

13 

ablaze 

89.9 

95.6 

97.8 

94.6 

99.3 

99.0 

97.0 

18 

avenue 

94.6 

93.2 

97.8 

98.0 

99.5 

98.5 

98 5 

21 

bench 

92.0 

90.8 

93.5 

92.2 

96 3 

92.0 

95.5 

22 

confess 

62.4 

86.0 

93.9 

92.2 

96.7 

99.0 

98.0 

25 

backward 

70 9 

88.4 

92.4 

87.6 

90.5 

95.0 

94.5 

26 

advertise 

69.0 

82.0 

93.1 

79.6 

88.8 

89.0 

89.0 

28 

combat 

59.6 

88.4 

92.4 

89.2 

97.4 

99.0 

99.0 

30 

blond 

62 . 4 : 

63.2 

92.8 

87.2 

96.0 

97.5 

98 0 

31 

broaden 

62.9 

83.2 

93.1 

94.6 

99.1 

97.5 

98.0 

32 

chubby 

64.6 

78.8 

93.5 

92.4 

95.8 

97.5 

98.5 

33 

concern 

65.1 

74.0 

93.5 

87.6 

94.7 

97.0 

95.5 

34 

cargo 

67 1 

89.2 

93.9 

84 0 

89,1 

92.5 

95.5 

35 

clutch 

60.2 

80.4 

92.4 

89.8 

94.0 

97.5 

97.6 

36 

awe 

29.4 

62.8 

82.4 

69.0 

83.5 

89.0 

86.0 

37 

aged 

53.8 

69.6 

83.5 

73.8 

85 8 

88.5 

90.0 

39 

arrive 

43.0 

68.8 

83.8 

63.8 

68 8 

73.5 

74.5 

40 

blunt 

41.0 

66.8 

84.5 

85.8 

92.3 

96.5 

94.0 

41 

accustom 

45.6 

62.4 

82.4 

52.0 

68.6 

70.0 

68.0 

42 

bade 

45.1 

« 

84.9 

72 6 

82.8 

84.5 

83.5 

43 

bog . 

40.2 

56.8 

84.2 

66.6 

79 3 

87.5 

88.0 

44 

cascade 

- 39.9 

56.8 

82.1 

65.6 

75.3 

87.5 

92.5 


* Omitted because 

of a 

misprint 

in test. 
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Grade 

City 

Number of 
Individuals 

6a 

N. Y. 

514 

N. Y. 

250 

9 

Mix. 

278 

9 + 10 
K 

500 

11 + 12 
K 

430 

12 

K, 

200 

12 

k; 

200 

46 

bray 

54.7 

62.0 

84.2 

79.4 

85.8 

93.0 

95.5 

50 

disembark 

50.4 

66.0 

82.4 

65.4 

81.4 

89.5 

95.0 

51 

conspire - - 

29.6 

70.8 

69,1 

61.0 

85.8 

94.5 

93.0 

54 

check ^ 

28.0 

67.6 

69.8 

50.6 

71.4 

SO >5 

77.0 

56 

cherish 

22.9 

40.4 

68.4 

48.4 

70.5 

72.0 

79.0 

57 

chirrup 

39.4 

56.8 

68.0 

66.0 

70.2 

73.0 

71.0 

58 

accessible 

29.8 

58.8 

68.4 

54.8 

82.3 

94.0 

94.0 

59 

dingy 

27.9 

53.6 

66.6 

71.0 

87.2 

93.5 

92.0 

61 

edible 

30.2 

42.4 

69.1 

57.2 

73.0 

91.5 

87.0 

62 

confound 

27.1 

47.2 

68.4 

43.2 

56.0 

52.5 

52.5 

63 

concur 

40.8 

56.4 

68.0 

61.6 

79.3 

83.5 

80 0 

64 

contact 

21.1 

54.0 

66.6 

57,2 

81.9 

85.0 

88.5 

66 

downcast 

21.3 

44.0 

51.5 

42.4 

61.4 

64.0 

61.0 

67 

pact 

19.2 

38.8 

49.3 

29.2 

63.7 

77.5 

77.0 

69 

audible 

5.0 

52.8 

49.3 

38.8 

60.7 

83.0 

80.5 

70 

solicitor 

45.0 

57.2 

49.3 

39.8 

47.0 

71.0 

68.5 

71 

beguile 


45.6 

48.6 

44.4 

59.1 

47.5 

51-0 

73 

dominate 


42.0 

49.3 

42.2 

70.7 

79.0 

82.5 

75 

average 


26.8 

48.2 

52.1 

61.6 

72.5 

67.5 

78 

behave 


35.6 

48.2 

39.8 

43.0 

70.0 

64.5 

79 

comely 


45.2 

48.6 

39.0 

42.8 

62.6 

64.5 

94 

cycle 


40.4 

31.7 

37.6 

50.7 

64.5 

67.0 

81 

action 


24.0 

29.5 

23.4 

33.3 

46.5 

42.0 

84 

avarice 


31.6 

29.2 

31.0 

42.3 

60.5 

58.5 

86 

bearing 


34.8 

32.4 

29.0 

34.9 

54.0 

43.5 

87 

allusion 


17.2 

31.7 

22.8 

32.1 

43.0 

43.0 

90 

dynasty 


22.8 

32.1 

23.8 

56.5 

70.5 

77.5 

91 

habitat 


15.6 

32.4 

26.0 

44.9 

54.0 

49.5 

92 

adversity 


25.2 

28.8 

22.6 

41.6 

67.5 

67.5 

93 

caprice 


22.0 

29.2 

21.2 

40.5 

55.0 

61.5 

105 

ignominious 


17.2 

17.6 

17.6 

30.7 

41.5 

42.0 

107 

chastity 


26.0 

16.9 

25.2 

38.4 

64.0 

64.0 

88 

gainsay 


22.0 

32.1 

18.8 

24.9 

30.0 

37.5 

89 

eclogue 


40.4 

30.9 

23.8 

23.3 

33.0 

35.0 

97 

cloistered 


12.0 

• 

10.8 

14.2 

31.0 

24.0 

98 

reciprocal 


10.0 

13.3 

11.0 

20.0 

26.0 

25.5 

99 

accolade 


16.0 

13.3 

11.8 

12.3 

15.0 

17.0 

100 

benighted 


11.2 

14.7 

7.5 

13.3 

16.0 

17.0 

102 

madrigal 


22.4 

17.6 

8.2 

11.4 

21.0 

28.0 

104 

pinnace 


15.6 

13.3 

8.4 

10.7 

14.5 

13.f 

106 

broach 


16.4 

18.3 

14.6 

27.4 

39.0 

34. 

110 

nectarine 


5.6 

18.3 

6.8 

14.9 

13.5 

12 


' Omitted because 

of a 

misprint 

in test. 
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Ninety tasks were chosen to represent harder words 
than level 7, and were used with one hundred college gradu- 
ates. From these ninety, four composites of ten each were 
chosen to be most alike in difficulty within a ten and most 
widely apart between tens. These four sets of ten were 
used with 240 college graduates who were also tested with 
levels 6 and 7. The results are shown in Table 29. We 
thus obtain level 8 of about the same difficulty as 7, and 
levels 9, 10, and 11 progressively harder. These levels 
from 1 to 11 are competent to measure word krtowledge 
from below the level of the average ten-year-old to far 
above the level of the average college graduate. 

Composites la, 2a, 3a, 4a, 5a, 6a, and 7a, of approxi- 
mately the same diflSculty as 1, 2, 3, 4, 5, 6, and 7, were con- 
structed by testing many pupils in grades 6, 8^, 9, 10, 11, 
12, and 100 college graduates with composites 1 to 7 and 
also with 240 new tasks, obtaining the percents succeeding 
with each of the 310 and selecting sets of ten from the 240 
to match sets 1, 2, 3, 4, 5, 6, and 7, respectively. The facts 
are shown in Tables 30 and 31. 

At the low end of the ability, the four sets A, B, C, and 
D shown below were constructed by selection from about 
twice as many on the basis of trials with 180 individuals 
16 years old or older of mental age from 2 to 4. The facts 
are shown in Table 32. 

Composites of ten intermediate between D and I were 
constructed on the basis of the ratings of about 160 single 
tasks by the consensus of twenty experts, and trials of these 
with a hundred adults of mental age 6.0 to 7.0, with 50 
feeble-minded individuals in the same educational “ class’' 
in an institution for the feeble-minded, with 101 pupils fif- 
teen years old or over in special classes in a large city, and 
with 162 pupils in grade 4B (second half). The facts con- 
cerning these word-knowledge tasks appear in Table 33. 

These composites intermediate in difficulty between 
V I) and V I are imperfect in three respects. The dif- 
ficulty of each single task element is not determined from 
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enough eases. The oral picture selection tests are not 
equated accurately enough with the oral word-selection 
tests. The difficulty of written word-selection tests has not 
been equated accurately against the difficulty of the same 
sort of test given orally. 

In general, we have devoted most of our work in the 
preparation of composite tasks to making effective instru- 
ments to measure altitude of Intellect CAVD from an alti- 
tude corresponding roughly to a mental age of ten up to 
very high levels. Our work with composites at lower levels 
has been aimed first at demonstrating that Intellect CAVl) 
can be measured at the altitude of low imbecility, and that 
we can, subject to certain limitations, locate an absolute 
zero point for intellect and so, by later studies which will 
bridge the interval between imbecility and our level I, at- 
tach approximate absolute values to all the levels. We have 
not been able to give adequate attention to the construction 
of CAVD composites to bridge this interval and our com- 
posites between D and I are not so well made as the easier 
and harder ones. 


Begin: 

1« bojhood 

2. churchman 

3. boyish 

4. cocoa 

5. bottomless 

6. assistant 

7. chauffeur 

8. dine 

9. blouse 
10. cafe 


11. dandruff 

12. abashed 

13. bethink 

14. comical 

15. apology 

16. clung 

17. amidst 


Level 1a 


1 cliildhood 2 mischief 3 hardihood 4 cap 5 cherub 
1 janitor 2 member of a church 3 elector 4 disciple 5 steeplejack 
1 naughty 2 male 3 impudent 4 like a boy 5 informal 
1 chocolate 2 a drug 3 chrysalis 4 biscuit 5 trivial 
1 artless 2 deeper 3 unreasonable 4 ultimate 5 without bottom 
1 orator 2 perseverant 3 progressive 4 at hand 6 helper 
1 carter 2 stove 3 hot water 4 coachman 5 automobile driver 
1 sprawl 2 visit 3 make a noise 4 have dinner 5 bespeak 
1 whisk 2 storm 3 below 4 pouch 5 waist 
I chaperon 2 theater 3 restaurant 4 f ask 5 festivity 

Level 2a 

1 ruffle 2 scamp 3 bald 4 dastard 5 disease of the scalp 
1 ashamed 2 overpowered 3 overlooked 4 bruised 5 lowered 
1 dream 2 molest 3 forget 4 ascertain 5 call to mind 

1 funny 2 coming 3 placid 4 typical 5 alert 

1 excuse 2 verdict 3 tribulation 4 conclusion 5 disease 
1 held fast . 2 part of a wheel 3 stung 4 part 5 nestled 

1 among 2 drenched 3 middle 4 lost 5 partly 


1 


. 2 
...3 



..6 
. 7 
. 8 
.. 9 
..10 


...11 
_.12 
...13 
....14 
.. 15 

.... 17 ' 
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18. baste 

19. causeless 

20. aster 

21. ballot 

22. rinse 

23. barge 

24. acquit 

25. cambric 

26. brawn 

27. appreciation 

28. alliance 

29. deceiver 

30. calculate 

31. childlike 

32. betwixt 

33. crafty 

34. outstrip 

35. available 

36. certify 

37. annihilate 

38. contentedly 

39. carcass 

40. console 


41. amen 

42. brawl 
43* debase 
4V^adventurous 
45 'j|dequate 
46. niable 

48. iKefactor 

49. Whought 

50. ;Arture 


51. t M ibe 

52. Ault 

53. a|BtT'3tion 
5A jjj^iance 
h JRiurlish 
515 . sexton 

Si, buckler 
*6. animosity 
9. conflagration 
K confidential 


1 sew. 2 list . 3 calico .. 4 wallow 5 dump ....18 

1 eventual 2 without reason 3 ineffective 4 highway . 5 faultless .....19 ^ 

1 flower 2 bitter 3 matin 4 star 5 guilder ...*..80 

Level 3a 

1 song 2 vote 3 ammunition 4 dance 5 award ......21 

1 scald 2 wash 3 smear 4 wrench 5 grin .... 22 

1 seaport 2 knock 3 tonnage 4 expansive 5 boat .....23 

1 do 2 free of blame 3 leave 4 aquatic 5 pipe 24 

1 brittle 2 linen 3 moccasin 4 leather 5 crochet . .. 25 

1 strength 2 brood 3 brine 4 burnt 5 bolster .....26 

1 forbearance 2 accompUshmeut 3 speech 4 sympathetic 

recognition 5 sermon .. 27 

1 league 2 enchantment 3 slander 4 hypocrisy 5 assembly . . 28 

1 detective 2 illusion 3 sx)y 4 cavalier 5 cheat .....29 

1 marvel 2 administer 3 plaster 4 reckon 5 convene ...30 

Level 4a 

1 innocent 2 saucy 3 foolish 4 piteous 5 affectionate .. 31 

1 confused 2 braided 3 between 4 bewitched 5 pinched .....32 

1 meager 2 difficult 3 adjacent 4 sly 5 artistic . .33 

1 subside 2 outer edge 3 outsklrt 4 satiate 5 out-run ... 34 

1 hidden 2 at hand 3 economical 4 lamentable 5 useful . 35 

1 exhort 2 ascertain 3 boast 4 fuse 5 assure ......36 

1 dead 2 crucify 3 enamor 4 nihilist 5 destroy ,37 

1 fully 2 heretofore 3 without a stop 4 cheerfully 5 massy ... 88 

1 mold 2 body 3 cargo 4 rind 5 hold of a ship ....39 

1 alone 2 qualify 3 visit 4 thin sole 5 soothe 40 

Level 5a 

1 SO be it 2 hymn 3 proverb 4 farewell 5 communion ....41 

1 pouch 2 roast 3 hoot 4 quarrel 5 lie at length ... 42 

1 degrade 2 base 3 chastise 4 blaspheme 5 unfounded .... 43 

1 clamorous 2 casual 3 bold 4 travel 5 advancing .... 44 

1 capricious 2 conscientious 3 enough 4 added 5 water supply .....46 

1 tractable 2 trusty 3 passionate 4 pleasing 5 odious .... 46 

1 league 2 associate 3 council 4 factor 5 navigator ...47 

1 patron 2 churchman 3 tourist 4 sexton 5 advantage ...48 

1 perhaps 2 credulous 3 forget 4 bewildered 5 considered ... 49 

1 through 2 precipice 3 opening 4 raiment 5 opportunity ....SO 

Level 6a 

1 attribute 2 pertain 3 clerk 4 write 5 upbiaid .....51 

1 defeat 2 blame 3 failure 4 libel 5 displace .... 52 

1 ghost 2 insurrection 3 apparent 4 farce 5 apparel ... 63 

1 request 2 adjustment 3 conformity 4 device 5 pliant .....54 

1 craven 2 rude 3 reckless 4 contemptible 5 envious .....65 

1 cube 2 janitor 3 compass 4 archbishop 5 six singers 66 

1 keel 2 servant 3 stag 4 shield 5 acr^per .....57 

1 hatred 2 animation 3 disobedience 4 diversity 5 friendship .....58 

1 carnival 2 celebration 3 decoration with flags 4 contagion . . 5 fire ., 59 
1 respectable 2 secure 3 sensitive 4 secret 5 confident .....60 
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61 ecrivener 
62. beaker 
62. emanate 

64. landau 

65. amaranthine 

66. athwart 

67. conscientious 
66. ingenuous 

69. betimes 

70, lambrequin 


Level 7a 

1 searcher 2 forger 3 chaplain 4 clerk 5 sceptic 
1 cup 2 binnacle 3 beak 4 slanderer 5 bottle 
1 populate 2 free 3 prominent 4 rival 5 come 
1 pier 2 coach 3 postern 4 gable 5 headdress 
1 jubilant 2 bitter 3 maritime 4 ungracious 5 purple 
1 alongside 2 above 3 alert 4 across 5 thwarted 
1 guilty 2 cautious 3 efficient 4 good 5 knowing 
1 ungenerous 2 unseldsh 3 dull 4 fiank 5 unthinking 
1 hereby 2 sometimes 3 meantime 4 early 5 now and then 
1 knapsack 2 drapery 3 raw wool 4 matting 5 chandelier 


61 
62 
.. 63 
. 64 
. 65 
.. 66 
. 67 
..68 
. 69 
70 


TABLE 29 

PEHMILIiBS COEBECT IN THE SINGLE TASKS OF WOED KNOWLEDGE 
10 — Composite Tasks 8, 9, 10 and 11 



T.C. Orad. 
n = 100 

L. Grad* 
n = 240 


T.C. Grad. 
n=100 

L.Grad. 
n = 240 


8 



10 




1. monomania 

550 

392 

1, shrievalty 

250 

283 


2. saturnalian 

520 

375 

2. sessile 

210 

179 


3. pristine 

510 

421 

3. teleological 

210 

221 


4. quaternion 

540 

346 

4. peccancy 

210 

358 


5, predatory 

520 

571 

5. cacophony 

240 

413 


6. persiflage 

500 

521 

6. pediment 

250 

254" 


7. encomium 

480 

600 

7. licentiate 

190 

154 


S. abattoir 

480 

613 

8. ambulatory 

220 

317 


9. meticulous 

510 

658 

9. murrain 

230 

133 

X 

10. largess 

9 

500 

429 

10. cantilena 

11 

230 

288 

2 

3 

A 

1. radial 

400 

408 

1. saltatory 

190 

121 

S 

2. sequestrate 

350 

529 

2. amerce 

110 

154 

6 

3, tactility 

360 

204 

3« distrain 

130 

458 

7 

4. apogee 

320 

363 

4. besom 

090 

154 

8 

5. nugatory 

820 

525 

5. rhodolite 

090 

188 

9 

, 10 

6. sedulous 

350 

363 

6. rune 

130 

112 

7. umbel 

350 

129 

7. hermeneutic 

100 

021 


8. asseveration 

340 

254 

8. devolution 

070 

046 

.. 11 

9. abjure 

.. 340 

342 

9. palindromic 

100 

112 

12 

10. auricular 

320 

321 

10. carmagnole . 

.. 120 

120 

. 18 
.14 


16 

17 
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TABLE 30. 

Pebmillies Ck>BRECT roB Each SinqijE Word or the Seven 10-Wobd Cokpositb 
Tases 1a, 2a, 3a, 4a, 5a, 6a, 7a in Each Op Vabiotjs 
Gbottpb op Individttals. 




6b 

11 = 1^9 

6c 

n = 105 

8% 

250 

9k 

306 

10k 

331 

Ilk 

224 

12k 

195 

1 

boyhood 

777 

990 

904 

917 

927 

933 

933 

2 

churchman 

820 

971 

928 

933 

960 

964 

970 

3 

boyish 

777 

942 

932 

891 

940 

946 

964 

4 

cocoa 

805 

990 

848 

911 

921 

937 

949 

5 

bottomless 

683 

933 

935 

968 

982 

982 

979 

7 

assistant 

640 

895 

952 

952 

960 

982 

985 

9 

chauffeur 

604 

942 

976 

968 

976 

991 

979 

10 

dine 

626 

933 

952 

952 

972 

996 

990 

13 

blouse 

604 

914 

956 

968 

966 

996 

990 

15 

cafe 

546 

933 

932 

981 

976 

991 

990 

11 

dandruff 

590 

790 

896 

965 

969 

991 

990 

16 

abashed 

661 

628 

752 

757 

828 

812 

872 

17 

bethink 

460 

752 

892 

863 

886 

875 

923 

22 

comical 

554 

809 

952 

964 

985 

996 

995 

23 

apology 

446 

834 

904 

912 

921 

937 

954 

24 

clung 

496 

866 

928 

967 

966 

991 

970 

31 

amidst 

446 

781 

856 

843 

892 

914 

923 

32 

baste 

446 

743 

640 

824 

857 

914 

913 

34 

causeless 

410 

790 

820 

819 

914 

954 

970 

39 

aster 

417 

657 

532 

889 

950 

946 

659 

33 

ballot 

424 

514 

756 

771 

824 

825 

816 

35 

rinse 

388 

581 

676 

637 

683 

749 

852 

42 

barge 

395 

638 

836 

752 

737 

888 

831 

45 

acquit 

453 

457 

724 

523 

647 

852 

841 

47 

cambric 

460 

343 

504 

706 

747 

861 

887 

58 

brawn 

316 

486 

736 

569 

700 

834 

846 

59 

apprec ration 

374 

571 

708 

676 

786 

847 

821 

61 

alliance 

244 

447 

728 

667 

7|28 

830 

826 

64 

deceiver 

252 

609 

732 

775 

721 

812 

826 

86 

calculate 

093 

371 

720 

596 

728 

843 

836 

36 

childlike 

496 

324 

356 

500 

528 

602 

718 

46 

betwixt 

230 

343 

464 

572 

631 

772 

785 

52 

crafty 

273 

457 

752 

542 

583 

669 

657 

60 

outstrip 

244 

324 

660 

539 

670 

727 

713 

67 

available 

173 

257 

504 

494 

715 

852 

852 
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TABLE 30 — Continued. 




6b 

n = 139 

6c 

n = 105 

8% 

520 

gk 

306 

10k 

311 

Ilk 

224 

12k 

195 

68 

certify 

108 

447 

532 

543 

570 

683 

677 

78 

annihilate 

201 

171 


507 

667 

825 

841 

80 

contentedly 

302 

486 

588 

549 

686 

754 

785 

94 

carcass 

144 

352 

580 

549 

615 

731 

881 

113 

console 

065 

228 

556 

509 

663 

785 

821 

60 

amen 

237 

466 

564 

425 

480 

629 

559 

54 

brawl 

266 

305 

568 

350 

441 

598 

636 

79 

debase 

273 

267 

432 

294 

438 

665 

682 

84 

adventurous 

122 

257 

432 

399 

486 

500 

584 

89 

adequate 

187 

114 

336 

363 

425 

598 

657 

93 

amiable 

209 

238 

448 

363 

460 

624 

667 

100 

ally 

201 

219 

416 

366 

441 

611 

672 

103 

benefactor 

173 

314 

384 

355 

409 

558 

652 

108 

bethought 

137 

152 

520 

359 

502 

549 

616 

109 

aperture 

093 

133 

416 

342 

460 

566 

616 

63 

ascribe 

345 

324 

256 

275 

316 

317 

416 

69 

default 

108 

219 

256 

271 

316 

352 

390 

85 

apparition 

151 

124 

360 

164 

219 

415 

605 

88 

appliance 

165 

162 

224 

157 

267 

406 

498 

101 

churlish 

230 

162 

292 

229 

283 

312 

359 

107 

sexton 

216 

162 

300 

228 

332 

379 

462 

112 

buckler 

165 

228 

220 

231 

267 

526 

374 

125 

animosity 

065 

124 

364 

176 

264 

368 

482 

137 

conflagration 

022 

048 

260 

160 

293 

459 

451 

138 

conddential 

124 

057 

216 

121 

216 

357 

457 

C53 

scrivener 




058 

146 

158 

185 

C73 

beaker 




106 

158 

231 

431 

0 76 

emanate 




067 

091 

098 

164 

C79 

landau 




080 

101 

133 

190 

0 83 

amaranthine 




102 

126 

150 

159 

0 88 

athwart 




067 

101 

197 

113 

0 89 

conscientious 




061 

126 

115 

195 

090 

ingenuous 




128 

154 

171 

195 

093 

betimes 




054 

032 

051 

082 

C05 lambrequin 




096 

066 

098 

149 


* Omitted because of misprint in test, 
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the oohstruotion of 10-composite tasks in sentence com- 
pletion, ARITHMETICAL PROBLEMS, AND THE UNDER- 
STANDING OF SENTENCES AND PARAGRAPHS 

The lO-composites for C, A, and D were constructed by 
the process of trying many single tasks with various groups 
and selecting tasks of similar difficulty, which has been de- 
scribed and illustrated in the case of V. Only the main re- 
sults will be presented here. They are in the form of tables 

TABLE 31. 

Pekmillss Obtaining Five oe More Right out or Ten in the V^cabiteart 
Composites 1, 1a, 2, 2a, 3, 3a, etc. 



Grade 5% 

N. Y. 
n= 148 

8V4 

N.Y. 

250 

9 

K 

1089 

10 

K 

723 

11 

K 

769 

12 

K 

643 

V 1 

993 

980 

993 

996 

997 

994 

V la 

993 

976 

993 

993 

999 

997 

V 2 

905 

920 

089 

989 

996 

994 

V 2a 

959 

972 

995 

991 

999 

lOUO 

V 3 

615 

740 

913 

924 

967 

975 

V 3a 

601 

864 

914 

929 

977 

987 

V 4 


645 

749 

801 

936 

946 

V 4a 


618 

764 

839 

931 

962 

V 5 


440 

428 

560 

748 

824 

V 5a 


448 

473 

604 

801 

846 

V 6 


152 

129 

290 

473 

560 

V 6a 


236 

183 

299 

480 

593 

V 7 


044 

017 

030 

061 

107 

V 7a 


060 

017 

031 

104 

137 


giving the percent of successes for each single task in each 
group. The constitution of the group sometimes varies 
within a table, because sometimes in a certain group some 
tasks would be assigned to only a part of the group. Where 
this is the case, the fact is noted by printing the new n in 
the body of the table. The n at the top of a column applies 
to all entries in that column unless a second n appears in 
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the colxuxm. If a second n appears in the column, it applies 
to all entries below it unless a third n appears ; and so on. 
Percents are strictly comparable only where they are for 
the same n. 

The sentence-completion 10-composites are A, B, C, D, 
E, F, G, I, J, K, L, M, N, O, P, and Q. The main facts con- 
cerning these are shown in Tables 34, 35, and 36. We also 


TABLE 32. 

Percxnts Corsect in the Singi<e Tasks oe Wokd Knoweedoe: Coupositb 
Tasks A, B, C and B. 180 Adtjdt Ihbeciles. 



11 = 100 

II 

00 

o 


n = 100 

n = 80 

A 1 

76 

81 

C 1 

26 

36 

2 

71 

79 

2 

25 

15 

3 

74 

75 

3 

24 

31 

4 

76 

80 

4 

24 

49 

5 

76 

89 

5 

23 

19 

6 

73 

85 

6 

22 

22% 

7 

72 

73 

7 

21 

30 

8 

72 

79 

8 

20 

24 

9 

67 

77% 

9 

21 

25 

10 

67 

76 

10 

21 

30 

B 1 

48 

55 

B 1 

12 

21 ' 

2 

49 

52% 

2 

15 

9 

3 

51 

61 

3 

11 

9 

4 

46 

62% 

4 

14 

16 

5 

44 

42% 

5 

15 

12% 

6 

47 

54 

6 

9 

12% 

7 

40 

36 

7 

14 

17% 

8 

43 

50 

B 

9 

7% 

9 

41 

57% 

9 

6 

6 

10 

39 

56 

10 

4 

12% 


have certain provisional completion 10-composites I-J and 
R which will be nseful until better ones are constructed. 

Some of these composites and also some of the arith- 
metic and directions composites to be presently described 
could probably be improved by transfers of some elements. 
We have not made these transfers, because the gain would 
not be g^eat and the labor of recomputing the composite- 
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TABLE 33. 

Pjescen-ts Oobjusct in the Single Tasks op Wokd Knowlbiioe E, F, G, Aim H. 






O CO 

11=: 50 

P. M. 

n = 101 
Spec. 

US 162 

4B 

V E 1 

gasoline (picture selection) 

59 

100 



2 

crayon 

4 4 

4 4 

51 

64 



3 

tresses 

4 4 

4 4 

55 

20 



4 

refrigerator ** 

4 4 

48 

70 



5 

plume 

i 4 

4 4 

48 

96 



6 

entrance 

4 4 

4 4 

46 

32 



7 

porridge 

4 4 

4 4 

44 

74 



8 

hide 

4 4 

4 4 

42 

76 



9 

drummer 

4 4 

4 4 

35 

68 



10 

ram 

44 

4 4 

38 

60 



V F 1 

rock (verbal selection) 

26 

94 

94.1 

98.1 

2 

people 

4 4 

4 4 

25 

92 

85.1 

97.5 

3 

large 

4 4 

4 4 

23 

90 

90.1 

96.9 

4 

heaven 

4 4 

4 4 

23 

88 

88.1 

92.6 

5 

speak 

4 4 

4 4 

22 

88 

84.2 

95.7 

6 

mountain 

4 4 

4 4 

22 

92 

96.0 

95.1 

7 

dark 

4 4 

4 4 

23 

94 

80.2 

93.8 

8 

kind 

4 4 

4 4 

21 

94 

77.2 

96.9 

9 

quiet 

4 4 

44 

20 

80 

77.2 

90.1 

10 

short 

4 4 

44 

20 

90 

79.2 

93.2 

V G 1 

good 

4 4 

4 4 

20 

70 

69.3 

87.7 

2 

still 

4 4 

4 4 

16 

70 

68.3 

95.1 

3 

warm 

4 4 

4 4 

21 

72 

81.2 

95.1 

4 

walk 

4 4 

4 4 

17 

72 

69.3 

95.1 

5 

behind 

4 4 

4 4 

17 

76 

73.3 

94.4 

6 

near 

4 4 

4 4 

16 

58 

72.3 

92.0 

7 

fast 

4 4 

4 4 

21 

74 

71.3 

95.7 

8 

once 

4 4 

4 4 

12 

62 

80.2 

98.1 

9 

sweet 

4 4 

4 4 

12 

58 

71.3 

90.1 

10 

bring 

4 4 

4 4 

7 

70 

71.3 

87.7 

V H 1 

love 

4 4 

4 4 

18 

62 

61.4 

84.6 

2 

lift 

4 4 

4 4 

6 

58 

61.4 

83.3 

3 

great 

44 

44 

8 

70 

63.4 

91.4 

4 

nation 

44 

44 

6 

58 

53.5 

71.6 

5 

space 

44 

44 

5 

38 

44.6 

73.5 

6 

none 

a 

4 t 

4 

60 

72.3 

90.7 

7 

every 

44 

44 

4 

38 

36.6 

70.4 

8 

before 

44 

4 4 

11 

56 

66.3 

84.6 

9 

pair 

44 

4 4 

0 

. 48 

47.5 

74.1 

10 

today 

t€ 

44 

7 

38 

49.5 

76.3 
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task scores and correlations which have been obtained from 
these 10-composites would greatly outweigh the gain. 

The arithmetical 10-composites were constructed in the 
same way by trial, selection, and retrial. It is difficult to 
secure large groups to take long experimental tests, and 

TABLE 34. 

Percents Stjcceedino with Each SinoiiE Task of Vasiohs IO-Compobites 
IN Two Groups op Adult Imbeciles. 



n 

= 100 

11 = 80 



ZL 

= 100 

tt 

GO 

O 

n 

= 100 

n = 80 

C A 

1 

78 

84 

A 

A 

1 

67 

64 

BA 1 

83 

87% 


2 

77 

79 



2 

53 

67% 

2 

85 

84 


3 

78 

81 



3 

75 

73 

3 

65 

69 


4 

82 

81 



4 

56 

67% 

4 

80 

80 


5 

68 

72% 



5 

67 

65 

5 

81 

86 


6 

59 

71 



6 

64 

73 

6 

85 

84 


7 

69 

74 



7 

67 

72% 

7 

72 

70 


8 

67 

62% 



8 

61 

67% 

8 

80 

86 


9 

63 

66 



9 

72 

64 

9 

73 

75 


10 

81 

73 



10 

60 

71 

10 

63 

73 

C B 

1 

46 

66 

A 

B 

1 

28 

50 

DB 1 

59 

65 


2 

65 

56 



2 

55 

46 

2 

43 

57% 


3 

57 

38 



3 

49 

42 

3 

58 

66 


4 

56 

55 



4 

49 

44 

4 

47 

50 


5 

53 

50 



5 

40 

54 

5 

40 

44 


6 

50 

45 



6 

55 

61 

6 

43 

75 


7 

56 

45 



7 

42 

42% 

7 

49 

44 


8 

51 

49 



8 

27 

49 

8 

42 

59 


9 

48 

56 



9 

44 

49 

9 

40 

74 


10 

40 

39 



10 

57 

40 

10 


47% 

0 0 

1 

26 

26 

A 

0 

1 

26 

24 

DC 1 

35 

34 


2 

37 

30 



2 

20 

24 

2 

24 

29 


3 

34 

35 



3 

34 

24 

3 

27 

36 


4 

44 

34 



4 

23 

29 

4 

17 

32% 


5 

36 

32% 



5 

38 

17 

5 

23 

19 


6 

37 

31 



6 

9 

20 

6 

19 

27% 


7 

27 

26 



7 

16 

30 

7 

20 

22% 


8 

31 

22% 



8 

21 

32 

8 

30 

35 


9 

30 

27% 



9 

15 

22 

9 

20 

24 


10 

22 

17% 



10 

21 

19 

10 

22 

25 

C D 

1 

35 


A 

D 

1 

24 

5 

BD 1 

16 

26 


2 

17 

19 



2 

18 

16 

2 

8 

10 


3 

12 




3 

11 

10 

3 

16 

25 


4 

19 

15 



4 

11 

11 

4 

10 

16 


5 

2 




5 

13 

14 

5 

16 

24 


6 

2 

6 



6 

23 

11 

6 

8 

22% 


7 

9 

4 



7 

22 

14 

7 

13 

30 


8 

1 

2% 



8 

12 

12 

8 

14 

17% 


9 

13 

14 



9 

21 

10 

9 

16 

17% 


10 

11 

1 



10 

17 

6 

10 

22 

9 
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TABLE 36. 

The PsKMHiijis Succeeding with Each Singue Task or Vabious 
lO-CoHPOSiTES OF Sentence Cohpdetions. 


Grade 

& = 

6% 

205 

8% 

250 

6 (1) 

' 61 

6(2) 

100 

6(3) 

107 

6(4) 

140 

17or + 
60 




C I 





1 

654 


787 

490 

738 

650 

933 

2 

654 


672 

610 

682 

593 

050 

3 

693 


443 

530 

748 

564 

900 

4 

746 

812 

607 

460 

757 

578 

1000 

5 

634 

804 

639 

380 

682 

564 

950 

6 

634 

824 

623 

460 

626 

585 

800 




11 = 162 

n =80 

11 = 75 

11 = 104 


7 

649 

840 

667 

362 

787 

510 

933 

s 

639 

796 

543 

500 

773 

779 

983 

9 

580 

800 

617 

413 

667 

519 

891 

10 

751 

832 

630 

587 

667 

603 

983 




C J 





1 

541 

580 

630 

437 

720 

712 

941 





n = 49 

11 = 52 

11 = 140 

11 = 59 

2 

605 

696 


531 

692 

690 

893 




11 = 375 

11 = 177 

n=186 

n = 248 


3 

498 

780 

501 

429 

613 

564 

783 

4 

532 

776 

481 

350 

600 

539 

983 

5 

493 

704 





925 

6 

341 

736 

443 

260 

495 

436 

958 

7 

463 

664 

377 

290 

542 

364 

717 

8 

358 

708 

278 

150 

547 

443 

891 

9 

294 

681 

279 

280 

402 

450 

900 

10 

206 

673 

352 

79 

414 

314 

925 




C K 










11 = 116 


a = 59 

1 

376 

364 



379 


890 




n = 184 

n = 49 

n = 53 

11 = 126 


2 

225 

432 

386 

122 

547 

238 

825 

3 

270 

644 

288 

41 

679 

262 

867 

4 

196 

336 

255 

184 

453 

333 

917 





11 = 49 

n = 52 

n = 140 


5 

377 

564 


61 

442 

155 

861 

6 

279 

568 






7 

353 

580 

123 

250 

293 

202 

808 

8 

225 

336 

185 

137 

280 

192 

683 

9 

152 

476 

246 

110 

299 

150 

983 

10 

191 

476 

180 

90 

280 

186 

908 


15 
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C L 

This is the least satisfactory lO-composite. It was used becausOi as a com- 
posite^ it filled a certain place. The 10 single tasks in order showed percents 
correct of 60, 59, 60, 54, 42^, 41, 15^, 11, 5% and % in a group of 200 pupils 
in grade 9. 


C M 

The 10 single tasks in order showed percents correct in a group of 200 
pupils in grade 9 of 22, 20, 36, 38, 30, 25, 28, 20, 26Vis, and 25. 


Grade 

n = 

5^2 

205 

8% 

250 

NS(1) 

100 

NS(2) 

100 

NS 

135 

NS 

87 

17or+ 

60 

17or+ 

28 

17or+ 

17 

S.Sch.. 

35 

10,11,12 

82 







C N 






1 

020 

200 

^70 

530 

482 

678 

830 

821 

882 

857 

350 

2 

083 

160 

690 

530 

467 

609 

817 

821 

824 

771 

386 

3 

005 

oso 

530 

420 

297 

483 

830 

964 

882 

886 

446 

4 

167 

240 

600 

570 

526 

713 

733 

786 

647 

914 

349 

5 

010 

108 

410 

470 

341 

540 

667 

964 

647 

829 

277 

6 

020 

164 

450 

500 

356 

506 

770 

893 

706 

714 

578 

7 

010 

92 

540 

550 

400 

575 

746 

893 

647 

857 

602 

8 

025 

100 

420 

400 

259 

506 

627 

964 

824 

600 

747 

9 

034 

192 

380 

580 

326 

609 

686 

893 

824 

771 

482 

10 

108 

148 

680 

710 

511 

759 

885 

893 

882 

913 

482 







o o 






1 



05)0 

160 

126 

253 

600 





2 



290 

220 

200 

264 

577 





3 



430 

380 

363 

391 

551 





4 



110 

150 

097 

172 

433 





5 



280 

210 

297 

3G8 

442 





6 



260 

330 

259 

448 

596 





7 



250 

230 

297 

253 

619 





8 



410 

490 

445 

586 

636 





9 



350 

430 

304 

425 

693 





10 



210 

190 

208 

345 

610 











C P 






1 



040 

020 

067 

000 






2 



150 

170 

193 

207 






3 



050 

040 

030 

034 






4 



100 

060 

059 

101 






5 



100 

050 

082 

149 
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TABLE Continued. 


Grade 5% 
11= 605 

8% N8(l) N8(2) 
520 100 100 

NS 

135 

NS 

87 

17or+ 

60 

17or+ 17or+ 
28 17 

S.Sch. 

35 

6 

120 

090 

119 

138 





7 

260 

240 

297 

310 





8 

100 

100 

082 

149 





9 

030 

090 

052 

080 





10 

110 

100 

111 

057 








C Q 





1 

080 

080 

082 

115 


750 

353 

171 

2 

000 

050 

015 

000 


429 

235 

114 

3 

000 

020 

008 

023 


671 

765 

229 

4 

030 

030 

015 

023 


500 

765 

257 

5 

020 

040 

030 

023 


500 

824 

257 

6 

000 

040 

000 

000 


429 

688 

171 

7 

020 

020 

015 

oil 


536 

529 

229 

8 

020 

030 

000 

000 


607 

471 

229 

9 

000 

010 

000 

000 


571 

706 

171 

10 

000 

010 

000 

000 


393 

706 

086 




C B 





1 

000 

000 

000 

000 


321 



2 

000 

000 

000 

000 


464 



3 

000 

010 

000 

000 


429 



4 

000 

000 

000 

000 


250 



5 

000 

000 

000 

000 


500 



6 

000 

000 

000 

000 


250 



7 

000 

000 

000 

000 


393 



8 

000 

000 

000 

000 


250 



9 

000 

000 

000 

000 


286 



10 

000 

000 

000 

000 


536 




when they have been secured, it is especially hard to obtain 
time enough to exhaust abilities in arithmetical problems. 
A patient adult may work for half an hour at a single prob- 
lem. We fear that the tasks M^hich happen to come late in 
the series as first printed showed fewer successes in our 
returns than they would have shown if they had been at- 
tempted first. In general, we feel less security that the per- 
cents of successes correspond closely with degrees of dif- 
ficulty in the case of the arithmetical problems than in the 
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TABIiE 37. 

The PEBMII.I.ES Stjcceedikq with Each SingiiE Task of Vabious lO-Ooicposrrss 

OF Abithueticaii Problems. 


A I 


Grade 

Sp 

Sp 

5% 

8% 

91 

9II 


60 

62 

189 

126 

246 

264 


1 

260 

346 

751 




2 

340 

192 

682 




3 

340 

288 

661 




4 

340 

404 

762 




5 

380 

308 

857 




6 

620 

673 

831 




7 

300 

365 

857 




8 

440 

423 

788 




9 

280 

250 

815 




10 

140 

135 

656 


A J 


1 



302 


646 

792 

2 



233 


626 

739 

3 



296 

818 

672 

708 

4 



545 

778 

846 

799 

5 



217 

627 

512 

538 

6 



370 

690 

667 

633 

7 



217 

643 

715 

610 

8 



344 

603 

768 

527 

9 



217 

540 

732 

564 

10 



328 

611 

821 

652 






A K 


1 



317 

532 



2 



143 

643 



3 



175 

571 



4 



307 

603 



5 



206 

524 



6 



058 

429 



7 



228 

540 



8 



196 

587 



0 



139 

579 



10 



105 

341 
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TABLE 37 — (Contmued) . 


Grade 

n = 

5% 

189 

8^ 91 

250 246 

9II N8(l) NS(2) NS 
264 100 100 135 

NS 

87 

17 

240 



A L 







1 

296 

544 







2 

296 

484 







3 

190 

448 







4 

206 

424 







5 

185 

396 







6 

153 

416 







7 

238 

516 







8 

148 

436 







9 

127 

376 







10 

201 

484 









A M 







1 


059 

174 

680 

670 

437 

563 


2 


093 

246 

730 

650 

481 

598 


3 


102 

220 

700 

650 

452 

644 


4 


089 

133 

720 

650 

496 

621 


5 


065 

140 

680 

630 

504 

736 


6 


053 

171 

580 

430 

341 

609 


7 


057 

129 

650 

600 

407 

701 


8 


057 

095 

600 

510 

348 

667 




A N 







1 




630 

480 

252 

391 

762 

2 




400 

390 

185 

322 

725 

3 




420 

290 

185 

379 

762 

4 




310 

260 

222 

230 

662 

5 




470 

370 

185 

333 

742 

6 




530 

420 

378 

517 

829 

7 




470 

450 

326 

437 

683 

8 




410 

430 

326 

471 

817 

9 




340 

300 

178 

310 

742 

10 




340 

260 

200 

379 

712 
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TABLE 37 — (Contimied) . 


Grade 
n = 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 


A O 


NS(l) NS(2) NS NS 17 
100 100 136 87 *240 


120 

120 

130 

030 

000 

030 

020 

010 

020 

020 


140 

130 

090 

040 

040 

040 

030 

040 

040 

050 


104 

081 

104 

022 

022 

022 

007 

044 

052 

059 


101 

230 

149 

034 

057 

115 

092 

057 

057 

092 


792 

779 

642 

467 

421 

079 

071 

042 

700 

600 


A P 


1 

2 

3 

4 

5 


546 
617 
400 
50 i 
650 


6 

7 

8 
9 

10 


579 

629 

502 

612 

0/5 


1 

2 

3 

4 

5 


A Q 


539 

343 

423 

502 

218 


case of C or V or D. However, the errors in this respect 
are probably very small in comparison with the difference 
in difficulty from Arithmetic A to Arithmetic P- 

The facts for the arithmetical 10-composites A, B, 

C, D, E, F, G, H, I, J, K, L, M, N, O, P, and Q are siven in 
Tables 34, 35, and 37. We have also certain provisional 
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TABLE 38. 

The Peecents Succeedinq with Bach Single Task or Various IO-Composites op 

Directions and Bsadings. 


rade 4 5 Ad 91 9II 10 11(1) 11(2) 11(3) 11(4) 12(1) 12(2) 12(3) Ad 

D= 162 311 44 246 236 100 100 100 100 63 100 100 84 44 









D % 







1 

51.9 

71.1 

75.0 

95.1 


96 

100 

96 

97 

98.4 




2 

26.5 

46.0 

63.6 

91.9 


95 

99 

92 

97 

96.8 




3 

5.6 

20.3 

56.8 

81.3 


96 

98 

96 

99 

100.0 




4 

34.6 

50.2 

77.3 

91.5 


92 

99 

98 

94 

98.4 










D 

1 







1 

37.7 

51.4 

65.9 

88.2 


85 

84 

94 

84 

88.9 




2 

56.2 

02.4 

65.9 

80.9 


91 

94 

89 

94 

90.5 




3 

6.2 

11,3 

38.6 

82.9 


89 

95 

90 

88 

87.3 




4 

35.2 

37.9 

59.1 

72,0 


92 

92 

92 

89 

96.8 




5 

40.1 

47.6 

63.6 

82.9 


91 

95 

94 

91 

98.4 




6 

26.5 

32.5 

52.3 

65,0 


82 

85 

93 

86 

98.4 




7 

24.1 

35.7 

34.1 

76.4 


92 

90 

90 

80 

96.8 




8 

41.4 

60.1 

65.9 

91.9 


95 

96 

93 

86 

05.2 




9 

6.2 

17.7 

50.0 

79.4 


83 

98 

93 

90 

94.0 




10 

46.9 

55.3 

47.7 

58.9 


81 

94 

91 

87 

90.5 










D 

o 







1 

17.3 

20.3 

50.0 

62.2 

52.6 

75 

82 

81 

79 

79.4 




2 

5.6 

8.7 

38.6 

47.6 

45.8 

69 

85 

83 

82 

81.0 




3 

25.9 

27.7 

52.3 

57.7 

53.S 

68 

88 

77 

73 

79.4 




4 

13.6 

9.6 

47.7 

41.9 

39.0 

70 

79 

79 

82 

79.4 




5 

5.6 

15.4 

45.5 

42.7 


79 

78 

85 

73 

88.9 




6 

16.0 

23.1 

47.7 

42.7 


77 

83 

77 

75 

74.6 




7 

4.3 

16.1 

52.3 

78.5 


74 

84 

76 

75 

84.1 




8 

39.8 

30.2 

43.2 

65.4 


78 

78 

83 

76 

81.0 




9 

9.3 

22.2 

40.9 

69.5 


78 

83 

72 

78 

76.2 




10 

16,7 

36.4 

40.9 

44.3 


60 

77 

77 

76 

76.2 










D 








1 

8.6 

33.8 


52.4 

48.3 

7t 

72 

78 

73 

79.4 

72 

71 

77.4 

2 

11.7 

15.4 


32.5 

38.6 

64 

77 

72 

70 

77.8 

66 

69 

79.8 

3 

15.4 

19.6 


58.1 

58.1 

63 

76 

76 

68 

73.0 

85 

69 

76,2 

4 

3.7 

3.9 


39.4 

24.2 

66 

71 

74 

72 

69.8 

79 

76 

85.7 

5 

9.3 

15.8 


59.3 


71 

72 

68 

78 

82,5 

76 

68 

75.0 
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TABLE 38 (continued). 


Grade 4 5 Ad 91 9II 10 11(1) 11(2) 11(3) 11(4) 12(1) 12(2) 12(3) 


n= 162 311 

44 246 

236 

100 

100 

100 

100 

63 

100 

100 

84 

6 


53.8 

67 

77 

74 

64 

67.7 

81 

79 

83.3 

7 


20.8 

56 

71 

71 

72 

69.2 

68 

69 

60,7 

8 


42.0 

54 

62 

73 

64 

69.2 

68 

67 

70.2 

9 


33.3 

61 

66 

64 

68 

69.2 

68 

68 

70.2 

10 


34.1 

48 

63 

66 

61 

60.0 

71 

67 

61.9 





D3 







1 

43.1 

37.7 

67 

66 

65 

57 

55.6 

70 

73 

61.9 

2 

31.7 

23.7 

64 

73 

71 

65 

77.8 

69 

66 

66.7 

3 

40.7 

22.5 

64 

67 

66 

54 

58,7 

60 

61 

63.1 

4 

46.7 

42.0 

62 

64 

68 

63 

63.5 

76 

78 

65.5 

5 

48.8 

38.6 

65 

62 

65 

62 

69.8 

74 

74 

72.7 

6 

43.5 

42.8 

60 

64 

69 

66 

65.1 

72 

80 

71.4 

7 

49.6 

39.4 

65 

64 

69 

66 

61.9 

67 

74 

73,8 

8 

22.4 

14.0 

69 

64 

64 

67 

74.6 

63 

71 

63.? 

9 

39.8 


56 

69 

70 

65 

77.8 

79 

72 

78. 

10 

41.1 

17.0 

59 

71 

72 

67 

76.2 

74 

64 

72.7 





D 4 







1 

38.6 

30.5 

44 

41 

54 

55 

57.1 

52 

53 

47.6 

2 

41.5 

28.8 

58 

50 

52 

51 

55.6 

48 

68 

47.6 

3 

42.3 

28.4 

45 

46 

49 

52 

57.1 

58 

73 

58.3 

4 

25.2 

19.9 

47 

49 

60 

56 

58.7 

65 

59 

65.5 

5 

42.3 

25.0 

47 

49 

44 

54 

52.4 

49 

48 

47,6 

6 

36.2 

25.4 

43 

55 

51 

53 

52.4 

56 

56 

54.8 

7 

45.1 

27.1 

46 

54 

51 

54 

44.4 

66 

64 

65.5 

8 

40.2 

25.9 

43 

55 

48 

54 

44.4 

61 

52 

60.7 

9 

19.5 

15.3 

40 

45 

52 

52 

50.8 

60 

45 

61.9 

30 

32.9 

12.6 

41 

49 

43 

42 

42.9 

41 

47 

38.1 





D4% 







1 



41 

42 

42 

46 

60.3 

56 

41 

59.5 

2 



51 

58 

45 

48 

47.7 

54 

45 

51.2 

8 



63 

50 

52 

42 

47.7 

67 

59 

58.3 

4 



47 

68 

57 

51 

58.5 

65 

60 

67.9 

5 



42 

59 

62 

61 

56.9 

63 

62 

54.8 
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TABLE 38 {continued) . 



9 

236 

10 11(1) 11(2) 11(3) 11(4) 12(1) 12(2) 12(3) 
100 100 100 100 63 100 100 84 

6 


42 

67 

57 

64 

52.3 

60 

71 

66.7 

7 


30 

52 

39 

37 

46.2 

55 

49 

46.4 

8 


33 

62 

48 

37 

43.1 

59 

51 

55.9 

9 


47 

49 

52 

61 

63.1 

52 

44 

48.8 

10 


47 

50 

59 

53 

61.5 

49 

58 

59.5 




D5 







1 


44 

46 

33 

40 

49.2 

54 

46 

55.9 

2 


32 

39 

39 

37 

25.4 

34 

32 

32.1 

8 


43 

47 

35 

47 

46.0 

55 

52 

64.3 

4 


31 

43 

39 

38 

36.5 

43 

41 

46.4 

5 


27 

30 

35 

32 

33.3 

53 

35 

42.9 

8 


27 

37 

31 

34 

34.9 

38 

22 

44.0 

7 


28 

40 

37 

25 

49.2 

45 

41 

40.5 

8 


25 

43 

30 

46 

46.0 

38 

41 

38.1 

9 


26 

42 

41 

37 

36.9 

47 

37 

44.0 

10 


41 

30 

38 

41 

33.8 

33 

33 

40.5 




D 6 







1 


16 

17 

24 

16 

14.3 

25 

22 

28.6 

2 


17 

21 

17 

18 

23.8 

21 

16 

21.4 

3 


30 

26 

21 

21 

32.3 

27 

24 

23.9 

4 


08 

16 

17 

09 

15.4 

17 

08 

13.1 

6 


19 

21 

20 

22 

16.9 

17 

24 

21.4 

6 


20 

24 

17 

05 

26.2 

27 

18 

28.6 




B 7 







1 


02 

00 

03 

02 

0.15 

00 

03 

02.4 

2 


02 

12 

06 

10 

40.0 

15 

21 

11.9 

8 


02 

03 

03 

05 

06,1 

11 

01 

04.8 

4 


00 

03 

03 

01 

01.5 

16 

07 

04.8 

6 


09 

08 

05 

04 

16.9 

13 

10 

09.5 

6 


05 

05 

08 

08 

12.3 

16 

09 

09.5 
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TABLE 38 (continued). 



N.S.(1) 

100 

N.S.(2) 

100 

N.S. 

135 

N.S. 

87 



D 5% 


1 

31 

37 

27.4 

46.0 

2 

27 

34 

26.7 

29.9 

3 

52 

44 

40.7 

39.1 

4 

54 

49 

34.8 

43.7 

5 

20 

40 

27.4 

23.0 

6 

37 

46 

38.5 

28.7 

7 

42 

42 

34.8 

31.0 

8 

41 

45 

34.1 

36.8 

9 

47 

40 

39.3 

26.4 

10 

31 

26 

25.9 

19.5 



D 6^ 



1 

29 

31 

23.7 

23.0 

2 

10 

9 

5.9 

5.1 

3 

24 

21 

18.5 

17.2 

4 

8 

8 

3.0 

5.7 

5 

7 

10 

10.4 

6.9 

6 

32 

27 

23.7 

11.5 


arithmetical 10-composites, IJ, J I, K I, and L I, which will 
be useful. 

In the case of comprehension of directions and para- 
graphs, we have composites of from six to ten elements A, 
B, C, D, E, F, G, 1, li 2, 2i, 3, 3J, 4, 4^, 5, 5^, 6, GJ, and 7, 
and have provisional composites II and 4. The facts con- 
cerning the difficulty of the constituent elements of these 
composites are given in Tables 34, 35, and 38. 

THE DTFFICULTY OF THE 10-COMPOSITES 

After the composites of ten have been obtained for sen- 
tence completions, arithmetical tasks, and understanding 
sentences, by such experimentation and selection as Las 
been described for the word-knowledge tasks, the difficulty 
of each composite in comparison with one or more others 
was measured in several groups of individuals. As many 
different composites were used in each group of individuals 
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as was feasible. It is bard to secure the cooperation of 
large groups in taking such long examinations as are neces- 
sary to put a large number of these composites in compari- 
son for the same group ; but in one way or another, we have 
accumulated a very large body of facts (shown in Tables 
39 to 50). 

We use these 10-composites to make 40-composites each 
containing IOC, lOA, lOV, and lOD.’^ They will also be 
available for special scales for sentence completion, arith- 
metical reasoning, vocabulary knowledge, and comprehen- 
sion of sentences and paragraphs. The arithmetical series, 
for example, is unquestionably a better instrument for mea- 
suring arithmetical ability of the problem-solving sort than 
has hitherto been available. 

In Tables 39 to 50 there are sometimes two forms of 
entry: (l)ercent successes) means the percent of the 

group in question having 50% or more of the single tasks 
right; **a distance” means the differmee in difficulty be- 
tween the 10-composite in question and a 10-composite of 
the same kind which exactly half of the group would suc- 
ceed with in the sense of having 50% or more of the single 
tasks correct. <j distance is in terms of the mean square 
deviation of the group in the ability measured by the 10- 
composite in question. The o distances even for the same 
group are not then strictly comparable, since the mean 
square deviation of the group in the ability measured by, 
say, C I, may not be identical with its mean square devia- 
tion in the ability measured by C J, or by A I, etc. Two 
10-composites of identical a distances will not, however, be 
far apart in difficulty. 

Minus ( — ) means easier than the median difficulty de- 
fined by the 10-composite which exactly half of the group 
succeed with; plus (-h) means harder than it. The form of 
distribution is arbitrarily assumed to be normal” in the 
case of all the abilities in all the groups. This is often 

7 In some cases the number is less than 10. Each single task is then given 
a weight BO that a perfect score would count 10. 
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erroneous and always doubtful, but will do no harm if its 
arbitrariness is kept in mind. The a distances are rough 
approximate measures convenient for comparison. The 
actual fact is always the %s. 

TABLE 39. 

The DipncuiiTT oe 10-Composite O— A, B, C, anb D; A — A, B, C, and D; 
V — A, B, C, AND D; AND D — A, B, C, and D, in the 
Case op 180 Adttlt iMBSciiiES. 



n 

%8 

100 80 

a distance 

100 80 


n 

100 80 

0 distance 
100 80 

c 

A 

84 

82% 

-1.19 

-1.13 

V 

A 

80 

81 

-1.03 

-1.07 


B 

65 

56 

- .51 

- .20 


B 

49 

57% 

+ .03 

- .25 


C 

35 

27% 

+ .45 

+ .68 


O 

14 

19 

+ 1.09 

+ .93 


D 

3 

0 

+ 1.59 

hi^n 


D 

1 

5 

+ 1.78 

+ 1.47 

A 

A 

69 

80 

- .65 

-1.03 

D 

A 

90 

86 

-1.45 

-1.27 


B 

45 

49 

+ .15 

+ .03 


B 

45 

67% 

+ .15 

- .59 


C 

15 

21 

+ 1.05 

+ .87 


C 

19 

27% 

+ .93 

+ .68 


D 

5 

5 

+ 1.47 

+ 1.47 


D 

12 

14 

+ 1.16 

+ 1.09 


TABLE 40. 

The Dippiculty op 10 Composites C — E, F, G and I, A — ^E, F, G 
V — Ej P, G A2n> H, AND D — ^E, F, G and H, in Vaeious Gboups, 
(o Distances abe Omitted from this Table.) 

AND H, 



%B 







100 

50 

101 


100 

50 

101 


MA.6 

Class 3 

special 


M.A.6 

Class 3 

Special 




Class 




CHass 

C E 

56 

94 

93 

V E 


94 


F 

25 

94 

88 

F 

23 

96 

96 

G 

7 

42 

74 

G 

15 

84 

85 

1 


20 

18 

H 

4 

64 

62 

A E 

55 

92 

100 

D E 

45 

96 

98 

F 

20 

90 

96 

F 

23 

92 

97 

G 

5 

74 

93 

G 

6 

58 

75 

H 


88 

73 

H 


44 

57 


The Sentence Completion and Arithmetical tasks were 
done in June, 1924 ; the Vocabulary and Directions-Reading 
tasks were done in September, 1924. We treat them to- 
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gether, since the differences due to an interval of less than 
three months are small. 

The following composites had fewer than 10 single 
tasks : Directions-Reading which had 4, 2^, which had 5, 
and 3, which had 6. 2 right, 3 right, and 3 right are used 
respectively, in these cases, instead of 5 right. 


TABLE 41. 

Thb Dimcxnwrr of IO-CSoufositsb O — P, G, I, J, and K, A — P, G, H, I, J, 
AND K, V— P, 6, H, 2, 3, AND 4, AND D— P, G, H, 1, 2, AND 
2%, IN Vabioxts Gaotrps in %b. 


nr: 

4B 

162 

5A 

125 

5B 

186 


4B 

162 

5A 

125 

5B 

186 

C P 

100.0 



V P 

98.8 



G 

83.3 



G 

98.1 



I 

35.2 

47.2 

67.6 

H 

87.0 

93.6 

100.0 

J 

22.2 

25.6 

44.6 

2 

50.6 

57.6 

75.3 

K 

1.2 

9.6 

6.4 

3 

25.9 

35.2 

53.8 





4 

5.6 

5.6 

9.7 

A P 

100.0 



D P 

93.8 



G 

98.8 



G 

82.1 



H 

96.3 

95.2 

98.4 

H 

75.9 

80.8 

91.9 

I 

72.8 

77.6 

86.6 

% 

36.4 

52.8 

68.3 

J 

6.8 

8.0 

25.3 

2 

3.1 

5.6 

10.2 

K 

0 

3.2 

15.6 

2% 

1.9 

0.8 

5.4 


Of the 53 adult students, only 45 were measured with 
the completion tasks, and only 28 of these attempted the Q 
and R composites. We have estimated as well as we can 
how the individuals in question would have succeeded if 
they had attempted all. 


THE COMBINATIOlSr OP 10-COMPOSITBS INTO 40-COMPOSITES 

These 10-composites were combined into 40-composites 
by putting together a C, an A, a V, and a D which, from the 
data at hand, seemed of nearly equal difficulty as com- 
posites. Some of the 10-composites were constructed espe- 
cially to fit others in this way. Into the history of the pro- 
cedure by which the final arrangement of the 40-composites 
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Average of the results for 2 and 2a. 
Average of the results for 3 and 3a. 
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TABLE 43. 

Tkb DiffictdIiTT of VA&iotre 10 -Compositbs in’ the Case of 44 Ai>uz#ts: 
RECEtriTS IN THE UNITED STATES ASMT. 



%8 




%a 



C F 

100.0 

A F 

100.0 

V F 

100.0 

D F 

100.0 

a 

86.4 

G 

100.0 

G 

100.0 

G 

95.5 



H 

100.0 

H 

100.0 

H 

88.6 

I 

77.3 

I 

95.5 

1 

100.0 

% 

79.6 

j 

70.5 

I-J 

70.5 

2 

90.9 

1 

61.4 

K 

50.0 



3 

81.8 

2 

47,7 





4 

61.4 

2% 

38.6 





5 

45.5 

3 

38.6 





6 

11.4 

4 

18.2 





7 

2.3 

5 

09.1 


TABLE 44. 

Diffictjdty of lO-CoMPOsiTEs Measured by the Permilles of Successes 
AND BY Distances + or — from the Median Difficulty for Grade 
8%> IN Units of the Mean Square Deviation of Grade 8% 

IN THE Ability Measured by the Composite. 


lO’Composites Permille s cr distance 


C I 

972 

-1.91 

J 

876 

-1.15!^ 

K 

472 

+ .07 

N 

72 

4-1.46 

A KI 

564 

- .16 

L 

440 

+ .15 

V 1 

980 

-2.05 

la 

976 

-1.98 

2 

920 

-1.41 

2a 

972 

-1.91 

3 

740 

- .64 

3a 

864 

-1.10 

4 

644 

- .37 

4a 

618 

- .30 

5 

440 

4" .15 

5a 

448 

4- .13 

6 

152 

4-1.03 

6a 

236 

4- .72 

7 

44 

+ 1.71 

7a 

60 

+ 1.55 

Inf. 3 (1) 

984 

-2.14 

4 (2) 

716 

- .57 

5 (3) 

500 

.00 

6 (4) 

356 

+ .37 

7 (5) 

324 

+ .46 

8 (6> 

200 

+ .84 
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TABLE 45. 

DimCTJI/TY OF lO-COMPOSITES Measttbsd bt the Peecekts of Two Gboofs 
(246 PmpiiiS IN Gbabe 9 and 264 Pupils in Grade 9) Succeeding wpdh 
Five of More of the Ten Single Tasks, and by Distances f or - 
FROU THE Median Difficulty for the Group in Units of the 
Mia.n Square Deviation of the Group in the Altitude of 
Whatever Ability the 10-Composite Measures. 


10-Composite 

Permille s 

91 

o distance 

Permille s 

9II 

a distance 

C I 

967 

-1.84 



IJ 

951 

-1.65 



J 

967 

-1.84 

936 

-1.52 

K 

805 

- .86 

689 

- .49 

lA 

350 

+ .39 

295 

+ .54 

Ml 

191 

+ .88 

178 

+ .92 

N 



30 

+ 1.88 

O 



0 


A I 

1000 

, . 


- 

I-J 

980 

-2.05 



J 

886 

-1.21 

773 

- .75 

J1 

943 

-1.58 

784 

- .79 

K1 

545 

- .11 

333 

+ .43 

K 

671 

- .44 

500 

.00 

L 

439 

+ .15 

258 

+ .65 

LI 



629 

- .33 

M 



167 

+ .97 

N 



72 

+ 1.46 

V 1 

996 

-2.65 

996 

-2.65 

la 





2 

967 

.84 

^v77 

-2.00 

2a 





3 

866 

-1.11 

826 

- .94 

3a 





4 

703 

- .53 

678 

- .45. 

4a 





5 

492 

+ .02 

405 

+ .24 

5a 





6 

150 

+ 1.04 

144 

+ 1.06 

6%a 



49 

+ 1.65 

7 

20 

+ 2.05 

23 

+ 2.00 

7s 



11 

+ 2.29 

1 In the case of the L and M completions 

with those pupils of Group 9 I 

who did not have 

time to do 

everything to 

their satisfaction, an estimated 

score was derived 

on the basis of what they did as far as 

they went and of 


what thej did with the completions of K. 
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TABIiF 45 — Continued. 


91 QH 


lO-Composite 

Permille 0 

or dietance 

Permille 0 

a distance 

D % 

992 

-2.41 



1 

951 

- 1.65 



2 

683 

- .48 



2% 

600 

- .25* 



3 

362 

-h .35 

205 

+ .82 

3% 

293 

+ .54 



4 

289 

-h .56 

133 

+ 1.11 

4% 





5 

126 

+ 1.15 

27 

■hl.93 

6 



23 

+ 2.00 

7 



0 




TABLE 46. 



Dipfictoty op 

lO'COMPOSITES 

Measured by 

THE a Distances + or - prom 

THB MEDIAX DifFICUI/TT OF A GIVEN GrAPE IN UNITS OF 

THE Mean 

Square 

Deviation op 

THE FOPUDATIOK OF THAT GRADE IN 

THE AbhjItt Measured bt the 

10 -coMPosiTBs. 



Grade 9 

Grade 10 

Grade 11 

Grade 12 


n = 1089 

11 = 723 

n=r769 

n = 643 

V 1 

2.457 

- 2.652 

- 2.748 

- 2.512 

2 

- 2.290 

- 2.290 

- 2.652 

- 2.512 

3 

- 1.360 

- 1.433 

- 1.838 

- 1.960 

4 

- .671 

- .845 

- 1.522 

- 1.607 

5 

+ .182 

- .151 

- .668 

- .931 

6 

+ 1.131 

+ .553 

+ .068 

+ .151 

7 

+ 2.120 

+ 1.881 

+ 1-546 

+ 1-243 


11 = 1041 

nr: 700 

n = 752 

n = 637 

V la 

- 2.457 

- 2.457 

- 3.090 

- 2.748 

2a 

- 2.576 

- 2.366 

- 3.090 


3a 

- 1,366 

- 1.468 

1.995 

- 2.226 

4a 

- .719 

- .990 

- 1.483 

- 1.774 

5a 

+ .068 

- .264 

- .845 

- 1-019 

6a 

+ .904 

+ .527 

+ .050 

- .235 

7a 

+ 2.120 

+ 1.866 

+ 1.259 

+ 1.094 



n = 1185 

n = 1053 

n = 742 

D 1 


- 2.409 

- 2.878 

- 2.652 

2 


- 1.398 

- 1.695 

- 1.812 

3 


- .690 

- .999 

- 1.170 

4 


- .055 

- .335 

- .516 

5 


•1- .542 

+ ,300 

- .065 

6 


+ 1.243 

+ .966 

+ .824 

7 


+ 2.170 

+ 1.896 

+ 1.774 


* 71.5% had 2 right out of 5. 48.4% had 3 right out of 5. 

16 
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TABLE 47. 

The DiFncuiiTT or Various IO-Composites Measured bt the Percent Suc- 
ceeding AND BY THE DISTANCE FBOU THE MEDIAN IN TERMS OF THE MEAN 
Square Deviation of the Group. 422 Normal School Seniors. The 
FORM OF Distribution is Assumed to be Normal.^' The Division 
into Two Groups of 150 and 185 is Approximately at Bandom. 

The Group of 87 Bkpresented a Somewhat Superior Selection 
AND took Certain Additional Tests. 


n = 

150 

185 

87 

422 

o 

150 

distances 

185 87 

422 

O M2 

98.7 

96.2 

98.9 

97.6 

-2.23 

-1.77 

-2.29 

-1.98 

MNs 

92.0 

80.5 

94.3 

87.4 

-1.41 

- .86 

-1.58 

-1.15 

N 

65.3 

45.9 

75.9 

59.0 

- .39 

+ .10 

- .70 

- .23 

Na 

54.0 

53.0 

69.0 

56.6 

- .10 

- .08 

- .50 

- .17 

No 

17.3 

19.5 

20.7 

19.0 

+ .94 

+ .86 

+ .82 

+ .88 

O 

18.7 

20.5 

35.6 

23.0 

+ .89 

+ .82 

+ .37 

+ .74 

OP 

04,7 

08.1 

04.6 

06.2 

+ 1.67 

+ 1.40 

+ 1.68 

+ 1.54 

P 

00.7 

04.3 

03.4 

02.8 

+ 2.46 

+ 1.72 

+ l.8a^ 

+ 1.91 

Pa 

13.0 

04.9 

03.4 

03.3 

+ 2.23 

+ 1.65 

+ 1.83 

+ 1.84 

Q 

00-0 

00.5 

00,0 

00.2 

00 

+ 2.58 

00 

+ 2.88 

Qa 

00,0 

00.0 

00.0 

00.0 

00 

00 

00 

00 

A 

90.7 

73.5 

83.9 

81.8 

-1.30 

- .63 

- .99 

- .91 

x._„ 

77.3 

51.9 

75.9 

65.9 

- .75 

- .05 

— 70 

- .41 

X. 

94.7 

85.4 

96.6 

91.0 

-1.62 

-1.05 

-1.83 

-1.34 

X„ 

84.0 

62.7 

79.3 

73,7 

- .99 

- .32 

- .78 

- .63 


72.7 

44.9 

69.0 

59.7 

- .60 

+ .13 

- .50 

- .25 

Y 

46.7 

22.2 

36.8 

33.9 

+ .08 

+ .77 

+ .34 

+ .42 

Ya 

45.3 

26.5 

42.5 

36.5 

+ .12 

+ .63 

+ .19 

+ .36 

Yz 

11.3 

06.5 

24.1 

11.8 

+ 1.21 

+ 1.51 

+ .70 

+ 1.19 

z* 

00.7 

03.2 

06.9 

03.1 

+ 2.46 

+ 1.85 

+ 1.48 

+ 1.87 

z. 

05.3 

02.2 

14.9 

05.9 

+ 1.62 

+ 2.01 

+ 1.04 

+ 1.56 

Z. 

Not taken off — ^so few got it. 




V 4a 

99-3 

95.7 

98.9 

97.6 

-2.46 

-1.72 

-2.29 

-1.98 

5a 

91.3 

80.5 

93.1 

87.0 

-1.36 

- .86 

-1.48 

-1.13 

6a 

72.7 

52.4 

73.6 

64.0 

- .60 

- .06 

- .63 

- .36 

6% 

44.7 

35.7 

50.6 

41.9 

+ .13 

+ .37 

- .02 

+ .20 

7 

18.0 

16.2 

20.7 

17.8 

+ .92 

+ .99 

+ .82 

+ .92 

I> 4%3 

96.7 

81.6 

90.8 

88.9 

-1.84 

- .90 

- .87 

-1.22 

5 



66.7 




- .43 


5% 

38.7 

22.7 

24.1 

28.7 

+ .29 

+ .75 

+ .70 

+ .56 

62 

54.7 

42.2 

34.5 

45.0 

- .12 

+ .20 

+ .40 

+ .13 


07.3 

04.3 

04.9 

05.7 

+ 1.45 

+ 1.72 

+ 1.65 

+ 1.58 

7 * 

10.7 

04.3 

08.0 

07.3 

+ 1.24 

+ 1.72 

+ 1.41 

+ 1.45 


2 Only six of the ten single tasks were used. 

8 Only nine of the ten single tasks were used* A person thus had to 
succeed with 55^% to score a success; hence the % is too low. 

4 This composite has only six single tasks. 



UEVmUS OF nyTRTiTJSCT 


217 


TABLE 48. 

Tax Di J ' fic ' UJ/ nr or IO-Compoeiites Mkasttbesd bt the PEBczaTS or 240 
CIolijBOE Obaduates and 100 Students or Education (CoutJEaB oa 
E'OBKAL School Graduates) Succeeding, and by Distances -t- or - 
PRoa THE Median Dirricui/TY for the Godleoe Graduates in Ques- 
tion, IN Units of the Mean Square Deviation op the College 
Graduates in Question in Altitude of Whatever Ability 
THE lO-CoaposiTE Measures in Each Case. 


n = 

% 

240 

100 

a distance 

240 100 



% 

240 

a distance 

100 240 100 

O K 

92.9 

95 

--1.47 

- 1.64% 

Vocab. 5 

99.6 

100 -2.65 


O 

80.8 

82 

- .87 

- .91% 

^ f 

5a 

100.0 

100 


P 

61.3 

56 

~ .29 

- .15 

< 4 

6 

99.6 

100 -2.65 


Q 

41.7 

44 

+ .21 

+ .15 

a 

6a 

94.6 

99 - 1.61 

-2.33 

B 

18.3 


4- .90 


( 4 

7 

68.4 

75 - .48 

- .67% 






4 4 

7a 

69.6 

77 - .51 

- -74 






4 4 

8 

53.3 

56 - .08 

- .15 






44 

9 

29.6 

31 + .54 

+ .50 

A N 

87.5 

91 

-1-15 

-1.34 

44 

10 

14.2 

10 + 1.07 

+ 1.28 

O 

82.1 

79 

- .92 

- .81 

44 

11 

1.7 

4 +2.02 

+ 1.75 

P 

67.5 

80 

- .45 

- -84 






Q 

39.2 


+ .27 












Inf. 

3(1) 


100 







4 4 

4(2) 


100 


D 4% 

96.3 


-1.79 


44 

5(3) 


100 


6 

85.9 


-1.08 


44 

6(4) 


97 

-1.88 

5% 


74 


- .64 

44 

7(5) 


81 

- .88 

6 

73.8 


f 

1 


44 

8(6) 


60 

- .25 

6% 


33 


+ .44 






7 

38,3 


+ .30 








As given to these groups A Q contained only 5 tasks, D 5 contained only 
8, D 6 contained only 6, D 6% contained only 6, and D 7 contained only 6. 
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TABLE 49. 

Difmculty of 10-Composites Measured by the Percents of 53 Adudt Stu- 
dents Succeeding, and by Distances + or — from the Median Difficulty 
FOR THE Group, in Dnitb of the Mean Square Deviation of the 
Group in the Ability Measured by the Composite. 



Permilles < 

j distance 

C M 

1000 


N 

962 

-1.77 

O 

887 

-1.19 

P 

774 

- .75 

Q 

453 

4- .12 

B 

245 

+ .69 

A J 

1000 


K 

1000 


L 

962 

-1.77 

M 

943 

-1.58 

2sr 

887 

-1.21 

o 

717 

- .57% 

p 

736 

- .63 


TABLE 50. 


DlFFlCUIiTY OF lO-COMPOSITES MEASURED BY THE PBRCENTS 

Students Succeeding. 

or 63 University 


N 

98.4 

V 6%a 

98.4 

O 

95.2 

7 

82.6 

P 

74.6 

8 

57.1 

Q 

33.3 

9 

20.6 

B 




M 


D 5 

93.7 

N 

81.0 

6 

77.8 

O 

69.9 

6% 

27.0 

P 

60.3 

7 

39.7 

Q 

14.3 
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was arrived at, we need not enter. The combinations made 
are as shown below : 


40-0oinposite 

0 

10-Composite 

A V 

D 

A 

A 

A 

A 

A 

B 

£ 

£ 

£ 

£ 

G 

C 

C 

C 

C 

D 

D 

D 

D 

D 

£ 

£ 

£ 

£ 

£ 

r 

£ 

F 

F 

F 

a 

G 

a 

G 

G 

H 


H 

H 

H 

I 

I 

I 

2 

% 

J 

J 

J 

3 

2 

K 

K 

K 

4 

2% 

L. 

L. 

£ 

5 

3 

M 

M 

M 

6 

4 

N 

N 

£■ 

6%a 

5 

O 

O 

O 

7 

6 

P 

P 

P 

8 

6% 

Q 

Q 

Q 

9 

7 

B 

B 


10 



The chief facts concerning the likeness in difficulty of 
the four 10-composites making each 40-eomposite are sum- 
marized in Table 51, which gives the percents succeeding 
for each of the four for each group which was tested with 
all four, and some other data. 

Not all of the facts of Tables 39—50 were available when 
the process of putting the 10-composites together to make 
40-composites was begun. If they had been, better com- 
binations could have been made. In some cases a 10-com- 
posite which is itself inferior is used because its difficulty 
as a composite fits a certain place. Such happened with 
Co L. In some cases 10-composites which seemed nearly 
enough equal in difficulty to belong well together, were 
found from wider experience not to be very closely alike, 
but the discovery was made too late to allow amendments 
to be made. 
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TABLE 61. 

SUUMABT OF THK FACTS OONCERNmO THE DlFFIOUiyr'Z' OF THE FOUB 
10-COUPOSITES COKSTITUTINO EACH 40-COlfPOSITE. 



C 

A 

V 

D 

Group 

n 

A 

84 

69 

80 

90 

im. 3 

100 


82% 

80 

81 

86 

€€ 

80 

B 

65 

45 

49 

45 

€€ 

100 


56 

49 

57% 

67% 

it 

80 

O 

35 

15 

14 

19 

it 

100 


27% 

21 

19 

27% 

it 

80 

D 

3 

5 

1 

12 

4 I 

100 


0 

5 

5 

14 

it 

80 

E 

56 

55 


45 

im. 6 



94 

92 

94 

96 

f. 



93 

100 

• 

98 

Bp. 


P 

25 

20 

23 

23 

im. 6 



94 

90 

96 

92 

f. 



88 

96 

96 

97 

sp. 



100 

100 

99 

94 

4 



100 

100 

100 

100 

ad. 


O 

7 

5 

15 

6 

im. 6 



42 

74 

84 

58 

f. 



74 

93 

85 

75 

sp. 



83 

99 

98 

82 

4 


H 



4 


im. 6 




88 

64 

44 

f. 




73 

62 

57 

sp. 




96 

87 

76 

4b 




95 

94 

81 

5a 


I 


98 

100 

92 

5b 



20 

14 

24 

12 

f. 



18 

20 

37 

. 

sp. 



35 

73 

51 

36 

4b 



47 

78 

58 

53 

5a 



68 

87 

75 

68 

5b 



83 

92 

93 

89 

6% 



97 

100 

97 

99 

91 
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TABliF SI (eontinued). 



O 

A 

V 

D 

Group 

n 

J 

22 

7 

26 

3 

4b 



26 

8 

35 

6 

5a 



45 

25 

54 

10 

5b 



43 

27 

55 

30 

5% 



97 

89 

87 

68 

91 



94 

77 

83 

— 

9 II 


K 

1 

0 

6 

2 

4b 



10 

3 

6 

1 

5a 



6 

16 

10 

5 

5b 



21 

16 


15 

5% 



81 

67 

70 

60 

91 



69 

50 

68 

- • 

9II 


L 

35 

44 

49 

36 

91 



30 

26 

41 

21 

9II 


M 

18 

17 

14 

13 

9II 




66 

64 

• ' 

N.S. 

422 

N 

3 

7 

5 

3 

9II 



59 

34 

43 

. 

N.S. 

422 


93 

88 


86 

17 Law 



95 

91 



17-18 Ed. 



98 

81 

98 

94 

Unir. 

63 

O 

23 

3 

18 

45 

N.S. 

422 


81 

82 

68 

74 

17 Law 



82 

79 

75 

.. 

17-18 Ed. 



95 

70 

83 

78 

Univ. 

63 

P 

3 

6 


7 

N.S. 

422 


61 

68 

53 

...... 

17 Law 



56 

SO 

56 

33 

17-18 Ed. 



75 

60 

57 

27 

Univ. 

63 

Q 

42 

39 

30 

38 

17 Law 



44 

. .. 

31 

... 

17-18 Ed. 



33 

14 

21 

40 

Univ, 

63 

B 

IS 


14 


17 Law 



— 

— 

10 

— 

17-18 Ed. 
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In general, however, these 40-composites are satisfac- 
tory. All the single tasks within any one of them are 
nearly enough alike in difficulty to prevent an individual 
for whom the composite as a whole is a suitable test, from 
being either bored or bewildered by any item of it. They 
rise in difficulty by steps which are small enough so that (as 
will be demonstrated later) a very finely graduated and rea- 
sonably precise measurement of altitude of Intellect CAVD 
is possible. In particular, composites J to Q furnish a very 
convenient series of defined and graded tasks. Each of 
them measures very nearly the same ability as the one 
below it or above it, and correlates with the score in the 
entire series of CAVD tasks nearly or quite as closely as 
its own self-correlation permits. 

The methods used for obtaining CAVD 40-composite 
tasks, each of which measures all of Intellect CAVD and 
nothing but it, save for a small chance error which can 

safely be corrected for by oi = Ot fti or by o, = <TtVrtit 2 , are 
applicable to any other form of intellect. We could have 
taken, in place of CAVD, Intellect OGAnS (O standing for 
the ability to give or select opposites, G standing for the 
ability to perceive the common element in n facts and select 
other facts containing it. An standing for the ability to per- 
ceive and apply relations as in an Analogies test, and S 
standing for the ability to put together to make a sensible 
total, a series of elements as in a dissected sentences test. 

Or we could have taken in place of CAVD, a non-verbal 
Intellect CBPF made up of Picture Completions, Geomet- 
rical Belations, Picture Sequences, and Forms to be cut up 
so as to produce specified parts. Or we could have taken an 
Intellect NIL made up of novel problems of the sort used 
in Burt’s Beasoning Test,® informational tasks, and tasks 
in learning.® Or we could have taken any combination of 
all these. 

• Shown in Chapter 2CVI. 

• Such as the substitation test of the Arrnj Beta, The National Intelli- 
gence Test, The Pintner-Non Verbal, sued other examinations. 
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In place of attaching approximately equal weight to 
each sort of tasks, we may use any specified weights. We 
may also weight the single tasks of the same sort (C or A 
or V or D) differently one from another, provided the sta- 
tistical consequences are allowed for consistently. 

We may use the same 'situation and task as an element 
in composites at different levels according to the quality of 
the performance. Thus to give ‘‘mean’' as the opposite of 
“grand” may count as a right in levels a, a + k, and 
a + 2k, whereas to give “little” may count as a right for a 
and a + k, but as a wrong for level a + 2k. 

The essentials are that whatever intellect we claim to 
measure be defined by an actual series of tasks, and that 
each composite task measure the ability. 

The difficulty of these CAVD 40-composites A, B, C, 
' • • Q is intellectual and it can be measured for each of 
them by means of the facts stated in Chapters II and IV, 
together with certain others. We shall carry through these 
measurements. But in order to do so, certain facts will be 
needed; and it will be most instructive to present these 
facts as parts of a general discussion of the meaning of tbe 
scores assigned in standard intelligence examinations. So 
the next two chapters will be in some respects a digression. 



CHAPTEE VII 


The Tbansfoemation of the Scobes of Standabd Examina- 
Tioisrs INTO Tebms of Scales with Equal Units 

THE METHOD OF TBANSFOBMATION ! ILLUSTBATED BY THE 
THOBNDIKE EXAMINATION AND ABMY ALPHA 

We have shown that the form of distribution of altitude 
of intellect measured in truly equal units is approximately 
Form A for any grade jwpulation from Grade 6 to college 
freshmen. We can then transmute the scores in any ex- 
amination which is closely symptomatic of intellect, into 
terms of truly equal units, if we have the distributions of 
scores in grade populations. Consider, for example, the 
facts of Table 52, which shows the essential steps in such a 
transformation and two further steps by which the mea- 
sures in truly equal units are adapted to the general scale 
of the original scoring. 

The first column is a scries of points on the scale of the 
original scores 54, 57, 60, 63, and so on. The second column 
gives the permille of individuals in the grade in question 
whose scores are below the point at which they are entered. 
Thus up to 54, there were 23 ; up to 57, there were 30 ; up to 
60, there were 43, and so on. The third column gives the 
distance below or above the central tendency for the grade 
in terms of the mean square deviation of the grade. The 
sign is — until the entry in column 2 passes 500, when it 
changes to -f-. The fourth column gives the successive dif- 
ferences, the entry on a line with 57 giving the true differ- 
ence between the score of 54 and the score of 57 ; the entry 
on a line with 60 giving the true difference between the 
score of 57 and the score of 60 ; and so on. These are the 
essentials. 

If we desire to put these differences in such shape as to 
compare each readily with the difference which it replaces, 
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TABLE 52. 

Thobkdikx Ex. H.S. Gkaos. P/lkt I, Forms D and N. Grade 12. n s 1527. 


1 

2 

3* 

4 

5 

6 

7 

Points 

on 

Original 

Scale 

Per- 
millo 
up to 
Stated 
Point 

z 

-or + 
Deviation 
from the 
Median 
in Equal 
Unite 

A 

Value of 

'Interval A/.05400** 
in Equal 

Units 

Corrected 
Points on 
Scale 

Golunm 6 
— Column 1 

54 

23 

- 1.995393 



54.77 

.77 

57 

30 

- 1.880794 

.114599 

2.12 

56.89 

- .11 

60 

43 

- 1.716886 

.163908 

3.04 

59.93 

- .07 

63 

59 

- 1.563224 

.153662 

2.84 

62.77 

- .23 

66 

75 

- 1.439531 

.123693 

2.29 

65.06 

- .84 

69 

95 

- 1.310579 

.128952 

2.39 

67.45 

-1.55 

72 

136 

- 1.098468 

.212111 

3.93 

71.38 

- .62 

75 

166 

- .970093 

.128365 

2.38 

73.76 

-1.24 

78 

214 

- .792619 

.177474 

3.29 

77.05 

- .95 

81 

263 

- .634124 

.158495 

2.93 

79.98 

-1.02 

84 

330 

- .439913 

.194211 

3.G0 

83.58 

- .42 

87 

391 

- .276714 

.163199 

3.02 

86.60 

- .40 

90 

463 

- .092878 

.183836 

3.40 

90. 

.00 

93 

536 

+ .090361 

.183239 

3.39 

93.39 

.39 

96 

600 

+ .253347 

.162986 

3.02 

96.41 

.41 

99 

656 

+ .401571 

.148224 

2.75 

99.16 

.16 

102 

720 

+ .582841 

.181270 

3.36 

102.52 

.52 

105 

770 

+ .738847 

.156006 

2.89 

105.41 

.41 

108 

807 

+ .866894 

.128047 

2.37 

107.78 

- .22 

111 

852 

+ 1.045050 

.178156 

3.30 

111.08 

.08 

114 

885 

+ 1.200359 

.155309 

2.88 

113,96 

- .04 

117 

916 

+ 1.378659 

.188300 

3.49 

117.45 

.45 

120 

934 

+ 1.506262 

.127603 

2.36 

119.81 

- .19 

123 

954 

+ 1.684941 

.178679 

3.31 

123.12 

.12 

126 

966 

+ 1.825007 

.140066 

2.60 

125.72 

- .28 

129 

980 

+ 2.053749 

.228742 

4.24 

129.96 

.96 


always 3 in this case, we may divide the entries of colninn 4 
by whatever o value is on the average equal to 1 in the orig- 
inal scores. The result of such a division appears as col- 
iinm 5. If we desire to use the true differences to form a 

* Computed from the Kell^*Wood table [Kelley, *23, p. 373 fif.]. 

** .054 is the average a value corresponding to a difference of 1.00 in the 
origmal scale. 
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true scale of the same general location and extent as the 
original scores, we can do so by taking any point of the 
original scale arbitrarily and replacing all the other points 
by the values derived from the true differences. Column 6 
shows the result of this operation, 90 being taken as the 
point of coincidence of the original and true values. 

Column 7 is simply a convenient way of showing how 
and how far the measures in truly equal units diverge from 

TABLE 53. 

Thorni>ik£ Ikt. Ex. H.S. Gbads. Pa&t I, Forms D and N ; Scores from 54 
TO 129 Corrected to be in Truitt E<)ual Units. 


Original 

Corrected 

Original 

Corrected 

Original 

Corrected 

54 

54.8 

80 

79.0 

106 

106.2 

55 

55.5 

81 

80.0 

107 

107.0 

56 

56.2 

82 

81.2 

108 

107.8 

57 

56.9 

83 

82.4 

109 

108.9 

58 

57.9 

84 

83.6 

110 

110.0 

59 

58.9 

85 

84.6 

111 

111.1 

60 

59.9 

86 

85.6 



61 

60.9 

87 

86.6 

112 

112.1 

62 

61.8 

88 

87.8 

113 

113.1 

63 

62.8 

89 

88.9 

114 

114.0 

64 

63.6 

90 

90.0 

115 

115.2 

65 

64.4 

91 

91.2 

116 

116.4 

66 

65.1 

92 

92.3 

117 

117.5 

67 

65.9 

93 

93.4 

118 

118.3 

68 

66.7 

94 

94.4 

119 

119.1 

69 

67.5 

95 

95.4 

120 

119.8 

70 

68.8 

96 

96.4 

121 

120.9 

71 

70.1 

97 

97.4 

122 

122.0 

72 

71.4 

98 

98.3 

123 

123.1 

73 

72.2 

99 

99.2 

124 

124.0 

74 

73.0 

100 

100.3 

125 

124,9 

75 

73.8 

101 

101.4 

126 

125.7 

76 

74.9 

102 

102.5 

127 

127.1 

77 

76.0 

103 

103.5 

128 

128.5 

78 

77.0 

104 

104.5 

129 

130.0 

79 

78.0 

105 

105.4 
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those by the original scores. In the illustration the diver- 
gences are small and irregular, so that for many purposes 
the original scores can be taken as approximately the scores 
in truly equal units. Finally, we may by interpolation ar- 
range a table giving the true-unit equivalents to any desired 
detail. When interpolating, one may also smooth the re- 
sults if he has good reason to believe that the irregularities 

TABLE 54. 

Aemt AIiPHA: Oraok 9: ii = 1721. 


Points on 
Original 
Scale 

Permillo 
up to 

Stated Point 

X 

- or *f 
Deviations 
from the 
Median in 
Equal Units 

A 

Value of 
Interval in 
Equal Units 

A/.0399 

55 

30 

- 1.880794 



60 

47 

- 1.674665 

.206129 

5.17 

65 

76 

- 1.432503 

.242162 

6.07 

70 

117 

- 1.190118 

.242385 

6.07 

75 

174 

- .938476 

.251642 

6.31 

80 

238 

- .712751 

.225725 

5.66 

85 

307 

- .504372 

.208379 

5.23 

90 

398 

- .258527 

.245845 

6.16 

95 

473 

- .067731 

.190796 

4.78 

100 

564 

+ .161119 

.228850 

5.73 

105 

630 

+ .331853 

.170734 

4.28 

110 

700 

+ .524401 

.192558 

4.83 

115 

755 

+ .690309 

.165908 

4.16 

120 

809 

+ .874217 

.183908 

4.61 

125 

856 

+ 1.062519 

.188302 

4.72 

130 

902 

+ 1.293032 

.230513 

5.78 

135 

936 

+ 1.522036 

.229004 

5.74 

140 

959 

+ 1.739198 

.217162 

5.44 

145 

971 

+ 1.895698 

.156500 

3.92 

150 

980 

+ 2.053749 

.158051 

3.96 


are due to the small size of the population rather than to 
inequalities in alleged units. In the case of the illustration, 
we present (in Table 53) the results of interpolation with- 
out smoothing. 

W'e have made the calculations necessary to obtain values 
in truly equal units for scores in Army Alpha, National A, 
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Otis Advanced Examination, Haggerty Intelligence Exami- 
nation Delta 2, Terman Group Test of Mental Ability, 
Myers Mental Measure, Pintner Non-Language Mental 
Test, I. E. R. Tests of Selective and Relational Thinking, 
Generalization and Organization, Army Examination a, the 
Trabue Mentimeter, and the Brown University Psycholog- 
ical Examination. The results appear in Tables 54 to 94. 

TABUB 55. 

Abut AXiPHA: O&abe 12: ii = 1387. 


Points on 
Original 
Beale 

Permille 
up to 

Stated Point 

X 

- or + 
Oeviations 
from the 
Median 

A 

Value of 
Interval in 
Equal Units 

A/.0411 

75 

18 

- 2.096927 



80 

40 

- 1.750686 

.346241 

8.42 

85 

52 

-- 1.625763 

.124923 

3.04 

90 

74 

- 1.446632 

.179131 

4.36 

95 

102 

- 1.270237 

.175395 

4.27 

100 

142 

- 1.071377 

.198860 

4.84 

105 

195 

- .859617 

.211760 

5.15 

110 

267 

-- .621912 

.237705 

5.78 

115 

343 

- .404289 

.217623 

5.29 

120 

422 

- .196780 

.207509 

5.05 

125 

497 

- .007520 

.189260 

4.60 

130 

581 

+ .204452 

.211972 

5.16 

135 

647 

+ .377234 

.172782 

4.20 

140 

711 

+ .556308 

.179074 

4.36 

145 

779 

+ .768820 

.212512 

5.17 

150 

838 

+ .986271 

.217451 

5.29 

155 

883 

+ 1.190118 

.203847 

4.96 

160 

924 

+ 1.432503 

.242485 

5.90 

165 

951 

+ 1.654628 

.222125 

5.41 

170 

968 

+ 1.852180 

.197552 

4.81 

175 

986 

+ 2.197286 

.345106 

8.40 


In each of these cases we have material from several 
grade populations, which it is necessary to combine. The 
methods of combination may be illustrated by Army Alpha. 
For the upper range of scores 75 to 175 or higher, we have 
three large grade populations: 1,721 in Grade 9, 1,387 in 
Grade 12, and 2,545 college freshmen (Ohio). Each of 
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these groups is submitted to the same treatment as was 
illustrated in the first four columns of Table 52. The re- 
sults appear in Tables 54, 55, and 56 (columns 1 to 4). The 
differences in truly equal units are then divided in each of 
the three tables by the o difference corresponding to the 
average raw Alpha score 'point for the interval 75 to 150 


TABTJ3 66. 

Akmy AiiPHA: Coli<eoe Fbbshmkk (Ohio): ii = 2545. 


Points on Pennille 
Original up to 

Scale Stated Point 

X 

- or + 
Deviations 
from the 
Median 

A 

Value of 
Interval in 
Equal Units 

A/.0337 

65 

20 

- 2.053749 



70 

28 

- 1,911036 

.142713 

4.23 

75 

36 

- 1.799118 

-111918 

3.32 

80 

51 

- 1.635234 

.163884 

4.86 

85 

70 

- 1.475791 

.159443 

4.73 

90 

94 

- 1.316519 

.159272 

4.73 

95 

125 

- 1.150349 

.166170 

4.93 

100 

162 

- .986271 

.164078 

4.87 

105 

206 

- .820379 

.165892 

4.92 

110 

257 

- .652622 

.167757 

4.98 

115 

309 

- .498687 

.153935 

4.57 

120 

379 

- .308108 

.190579 

5.81 

125 

442 

- .145900 

.162208 

4.81 

130 

495 

- .012533 

.132367 

3.93 

135 

565 

+ .163658 

.176191 

5.23 

140 

624 

+ .316003 

.152345 

4.52 

145 

701 

+ .527279 

.211276 

6.27 

150 

763 

+ .715986 

.188707 

5.60 

155 

818 

+ .907770 

.191784 

5.69 

160 

871 

+ 1.131131 

.223361 

6.63 

165 

906 

+ 1.316519 

.185388 

5.50 

170 

939 

+ 1.546433 

.229914 

6.82 

175 

961 

+ 1.762410 

.215977 

6.41 

180 

981 

+ 2.074855 

.312445 

9.27 

(that is. 

by (X,50 — 

X,„)/75). This 

interval is 

taken be- 


cause it is eouamon to all three of the grade populations, so 
that its use gives strictly comparable results from all three. 
These results appear as the last columns in Tables 54, 55, 
and 56, and again in Table 57, where they are averaged for 
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each 5-point interval of the original scores. The last col- 
umn of Table 57 gives the unreliability in terms of Ot - o 
(mean square error) of each average. We have rather re- 
liable measures in truly equal units from 75 to 155, and less 
reliable measures down to 55 and up to 175. 

TABLE 57. 

Abut Alpha: Obases 9, 12 and 13 (Coixege Fkeshmen) : Vaiues of Suc- 
cessive 5-Point IntervaiiS of the Obioinax. Scokes, in Equai. Units. 


Interval: Values in Equal Units 

Original Grade 9 Grade 12 Grade 13 Average *^t-o 

Score XL = 1721 n = 1387 n = 2545 


55 

to 

59 

5.17 



5.17 


60 

to 

64 

6.07 



6.07 


65 

to 

69 

6.07 


4.23 

5.15 

.66 

70 

to 

74 

6.31 


3.32 

4.82 

1.06 

75 

to 

79 

5.66 

8.42 

4.86 

6.31 

.87 

80 

to 

84 

5.23 

3.04 

4.73 

4.33 

.54 

85 

to 

89 

6.16 

4.36 

4.73 

5.08 

.45 

90 

to 

94 

4.78 

4.27 

4.93 

4.66 

.16 

95 

to 

99 

5.73 

4.84 

4.87 

5.15 

.24 

100 

to 

104 

4.28 

5.15 

4.92 

4.78 

.21 

105 

to 

109 

4.83 

5.78 

4.98 

5.20 

.24 

110 

to 

114 

4.16 

5.29 

4.57 

4.67 

.27 

115 

to 

119 

4.61 

5.05 

5.81 

5.16 

.29 

120 

to 

124 

4.72 

4.60 

4.81 

4.71 

.05 

125 

to 

129 

5.78 

5.16 

3.93 

4.96 

.44 

130 

to 

134 

5.74 

4.20 

5.23 

5.06 

.37 

135 

to 

139 

5.44 

4.36 

4.52 

4.77 

.28 

140 

to 

144 

3.92 

5.17 

6.27 

5.12 

.56 

145 

to 

149 

3.96 

5.29 

5.60 

4.95 

.41 

150 

to 

154 

4.32 

4.96 

5.69 

4.99 

.32 

155 

to 

159 


5.90 

6.63 

6.27 

.27 

160 

to 

164 


5.41 

5.50 

5.46 

.03 

165 

to 

169 


4.81 

6.82 

5.82 

.74 

170 

to 

174 


8.40 

6.41 

7.41 

.70 


For the lower range of scores (from 95 down) we have 
three grade populations (263, 281, and 321), though not so 
large ones as is desirable. They also are treated in the 
same manner as has been illustrated, up to the point of the 
differences in equal units, the results appearing in Tables 
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58, 59, and 60. These differences are then divided hy 
(Xss — X2 b)/ 60. The results of the divisions appear as the 
three last columns of Tables 58, 59, and 60, and again in 
Table 61, where they are averaged for each 5-point interval 
of the original scores. We thus have measures in truly 
equal units for all the range from 20 to 85, though not so 
reliable ones as we should like to have. As an addition to 
these determinations, we have used in a similar manner the 
records from 242 12th-grade boys, 393 12th-grade girls, and 
400 college freshmen. The results appear in Table 62. 

We have combined all these results from these three 
determinations, giving weights of 3 to the determination 
from 5,653 individuals (1,721 -f- 1,387 -j- 2,545), 1 to the de- 
termination from 865 individuals (263 -f- 281 -4- 321), and 1 
to the determination from 1,035 individuals (242 -f- 393 -f- 
400).* The results appear in Table 63. 

In Table 64 we give the results of interpolation with 
rather liberal smoothing. In the smoothing, we have been 
guided by three facts. First, the ups and downs from about 
55 to about 145 are quite irregular and, in view of the prob- 
able errors, may well be chances of the sampling. Second, 
the data of Table 61 for the interval 15 to 20 and the gen- 
eral drift of the Army results support the hypothesis that 
original scores from 10 to 20 will have a true-unit value 
even higher than our 13.67 for 20 to 30 ; so that the increase 
from 10.81 for 40 to 50, to 11.67 for 30 to 40, and 13.67 for 
20 to 30 may be considered real. Third, the facts of Table 
62 and the facts of Table 65 for Alpha scores of a college- 
graduate group show that the true-unit values per interval 
of 5 score-points continue, above 175, the rise shown in our 
table from 150 to 170. We take 100 as the point of coinci- 

^ Before combining tbem we have multiplied all the results from the 865 
determiiiatioii (Grades 5, 6, and 7) by 1.08, which puts their total value from 
65 to 95 on a parity with the total value from 65 to 95 of the 5623 series; we 
have also multiplied all the results from the 1035 determination by 1.03, whi<^ 
puts their total value from 110 to 155 on a parity with the total value from 
110 to 155 of the 5623 series. These multiplications serve to keep our systems 
of values consisteBt. 

17 
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TABLE 58. 

Arut AI.PHA : Obadb 5 : a = 263. 


PoiatB 

on 

Original 

Scale 

Pennille 
up to 
Stated 
Point 

X 

-or + 
Deviation 
from the 
Median 

A 

Value of 
Interval 
in Equal 
Units 

A/.0588 

15 

34 

- 1.825007 



20 

106 

-^1.248085 

.576922 

9.81 

25 

190 

- ,877896 

.370189 

6.29 

30 

293 

- .544642 

.333254 

5.67 

35 

395 

- .266311 

.278331 

4.73 

40 

494 

- .015040 

.251271 

4.27 

45 

589 

+ .224973 

.240013 

4.08 

50 

692 

+ .501527 

.276554 

4.70 

55 

768 

+ .732276 

.230749 

3.92 

60 

848 

+ 1.027893 

.295617 

5.03 

65 

905 

+ 1.310579 

.282686 

4.81 

70 

939 

+ 1.546433 

.235854 

4.01 

75 

970 

+ 1.880794 

.334361 

5.69 

80 

989 

+ 2.290370 

.409576 

6.96 

85 

996 

+ 2.652070 

.361700 

6.15 



TABLE 59. 




Army Alpha : Grade 6 : 

n=281. 




X 

A 


Points 

Permille 

— or + 

Value of 


on 

up to 

Deviation 

Interval 

A/.0551 

Original 

Stated 

from the 

in Equal 


Scale 

Point 

Median 

Units 


20 

14 

- 2.197286 



25 

39 

- 1.762410 

.435876 

7.91 

30 

89 

- 1.346939 

.415471 

7.54 

35 

132 

- 1.116987 

.229952 

4.17 

40 

214 

- ,792619 

.324368 

5.89 

45 

299 

- .527279 

.265340 

4.81 

50 

427 

- .184017 

.343252 

6.23 

55 

566 

- .166199 

.350216 

6.36 

60 

630 

+ .331853 

.165654 

3.01 

65 

737 

+ .634124 

.302271 

5.49 

70 

797 

+ .830953 

.196829 

3.57 

75 

836 

+ .978150 

.147197 

2.67 

80 

886 

+ 1.205527 

.227377 

4.13 

85 

939 

+ 1.546433 

.340906 

6.19 

90 

964 

+ 1.799118 

.252685 

4.68 

95 

985 

+ 2.170090 

.370972 

6.73 
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dence of the scale of the original scores and the scale of 
their equivalents in truly equal units. If we use the values 
of Table 63, we reach 65.92 as the equivalent of 65, and 
149.90 as the equivalent of 150. The .10 deficiency we sim- 
ply disregard and go from 100 to 150 by steps of 1.0. The 
.92 excess we distribute oVer the interval from 80 to 90, 

TABLE 60. 

Abmt Alpha: Grade 7: ii = 321. 


Points 

on 

Original 

Beale 

Permille 
up to 

Stated 

Point 

X 

- or + 
Deviation 
from the 
Median 

A 

Value of 
Interval 
in Dqual 
Units 

A/.04915 

23 

15 

-2.170090 



30 

22 

-2.014091 

.155999 

3.17 

35 

56 

- 1.589268 

A24823 

8.64 

40 

87 

- 1.359463 

.229805 

4.68 

45 

140 

-1.080319 

•279144 

5.68 

50 

196 

- .855996 

.224323 

4.56 

55 

290 

- .553385 

.302611 

6.16 

60 

371 

- .329206 

.224179 

4.56 

65 

449 

- .128188 

.201018 

4.09 

70 

539 

+ .097914 

.226102 

4,60 

75 

626 

+ .321278 

.223364 

4.54 

80 

732 

+ .618873 

.297595 

6.05 

85 

782 

+ .778966 

.160093 

3.26 

90 

826 

+ .938476 

.159490 

3.24 

95 

863 

+ 1.093897 

.155421 

3.16 

100 

910 

+ 1.340755 

.246858 

5.02 

105 

931 

+ 1.483280 

.142525 

2.90 

110 

963 

+ 1.786614 

.303334 

6.17 

115 

969 

+ 1.866296 

.079682 

1.62 

120 

984 

+ 2.144411 

.278115 

5.66 


which seems the most likely place for it to belong. From 65 
down and from 150 up, we increase the values following the 
general course of the results of our determinations. 

Two hundred and sixteen first-year law students,® all col- 
lege graduates, were tested with Army Alpha. The dis- 
tribution was as shown in Table 65. We do not know pre- 

a Omitting two foreign students. 
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cisely the form of distribution of intellect measured in 


truly equal units for such a group as this, but the facts 
given in Appendix VI make it safe to assume that the upper 
half of the distribution has a form not much unlike Form A. 
By Form A the values of the intervals from the median up 

are: 

180 to 184 

.4565a 

185 to 189 

.3089a 

190 to 194 

.3609a 

195 to 199 

.5586a 

200 to 204 

.6807a 


TABLE 61. 

Abut Alpha, Grades 5, 6 and 7. Values op Successive 5-Point Intervals 
OP THE ObIOINAL SCORES, IN EQUAL UNITS. 


Interval 

Original 

Score 

Grade 5 
n = 263 

Values in Equal Units 

Grade 6 Grade 7 

nr: 281 n = 321 Average 

0t - o 

15 to 19 

9.81 



9.81 


20 to 24 

6.29 

7.91 


7.10 

.50 

25 to 29 

5.67 

7.54 

3.17 

5.28 

1.04 

30 to 34 

4.73 

4.17 

8.64 

5.85 

1.15 

35 to 39 

4.27 

5.89 

4.68 

4.95 

.40 

40 to 44 

4.08 

4.81 

5.68 

4.86 

.38 

45 to 49 

4.70 

6.23 

4.56 

5.16 

.44 

50 to 54 

3.92 

6.36 

6.16 

5.48 

.64 

55 to 59 

5.03 

3.01 

4.56 

4.20 

.50 

60 to 64 

4.81 

5.49 

4.09 

4.80 

.33 

65 to 69 

4.01 

3.57 

4.60 

4.06 

.34 

70 to 74 

5.69 

2.67 

4.54 

4.30 

.72 

75 to 79 

6.96 

4.13 

6.05 

5.71 

1.18 

80 to 84 

6.15 

6.19 

3.26 

5.20 

.79 

85 to 89 


4.58 

3.24 

3.91 

.47 

90 to 94 


6.73 

3.16 

4.95 

1.26 

95 to 99 



5.02 

5.02 


100 to 104 



2.90 

2.90 


105 to 109 



6.17 

6.17 


110 to 114 



1.62 

1.62 


115 to 119 



5.66 

5.66 
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TABliE 62. 

Akmt Ai<pha: Gbases 12 and 13; Suppdeicentai. VALtrES or Sttccessive 
5'Point IntebtaiiS of the Obioinai. Scores, in Equad Units. 


Interval 

Original 

Score 

Grade 12 
Boys 

11 = 242 

Values in Equal Units 

Grade 12 Yale 

Girls Yresbman 

n = 393 n = 400 Average 

Ot - o 

75 to 79 

8.05 



8.05 


SO to 84 

2.20 

3.00 


2.60 

.28 

85 to 89 

2.13 

1.08 


1.61 

.37 

90 to 94 

5.36 

3.80 


4.58 

.55 

95 to 99 

1.80 

5.16 


3.48 

1.19 

100 to 104 

5.56 

5.84 


5.70 

.10 

105 to 109 

4.89 

6.05 


5.47 

.41 

110 to 114 

4.76 

5.83 

7.13 

6.91 

.66 

115 to 119 

5.01 

5.70 

3.52 

4.74 

.53 

120 to 124 

3.29 

4.56 

4.06 

3.97 

.30 

125 to 129 

6.14 

5.06 

5.47 

5.56 

.26 

130 to 134 

2.24 

5.57 

4.76 

4.19 

.82 

135 to 139 

4.26 

4.11 

2.94 

3.77 

.34 

140 to 144 

7.19 

4.71 

5.45 

5.78 

.60 

145 to 149 

5.35 

4.25 

4.66 

4.75 

.26 

150 to 154 

5.23 

3.09 

4.99 

4.44 

.55 

155 to 159 

6.64 

5.63 

6.13 

5.68 

.35 

160 to 164 

3.37 

6.52 

5.00 

4.96 

.74 

165 to 169 

6.58 

4.95 

5.86 

5.80 

.39 

170 to 174 

10.29 


7.96 

9.13 

.89 

175 to 179 



7.64 

7.64 


180 to 184 



7.95 

7.95 


185 to 189 



6.67 

6.67 

* 

190 to 194 



6.01 

6.01 



The values for the two intervals next below the median are : 


170 to 174 .31360 

175 to 179 .25850 

These last and the values for 180 to 189 will not be very 
different if any reasonable assumption is made about the 
form of distribution. 

Expressing the values for 190 and above in terms of the 
170 to 190 difference, which is 1.327626o, we have the fol- 
lowing results: 

190 to 194 = .272 
195 to 199 = .421 
200 to 204 = .513 
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TABLE 63. 

Finaii Ebtimatx or Belatite Valtjes or Abut Alpha Scobes in Equal 

Units. 



20 to 24 
25 to 29 
30 to 34 
35 to 39 
40 to 44 
45 to 49 
50 to 54 
55 to 59 
60 to 64 
65 to 69 
70 to 74 
75 to 79 
80 to 84 
85 to 89 
90 to 94 
95 to 99 
100 to 104 
105 to 109 
110 to 114 
115 to 119 
120 to 124 
125 to 129 
130 to 134 
135 to 139 
140 to 144 
145 to 149 
150 to 154 
155 to 169 
160 to 164 
165 to 169 
170 to 174 
175 to 179 
180 to 184 
185 to 189 
190 to 194 


13.67 


11.67 


10.81 


10.46 


10.14 


11.06 


10.37 


10.12 


10.27 


11.06 


11.23 


15.78 


16.06 
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TABLE 64. 

EQUIVAIiENTa FOB ABMY ALPHA SCOBES FROM 20 TO 170 IK A SCAUS WITH EQUAIi 
IJKITS, 1 OF THIS SCAUE EQUALLIKS .89/90 OF THE DIFFERENCE BETWEEN 
60 AND ISO OF THE ObIOINAIi ALPHA SCORES, OB APPROZIMATEDT 1/100 
OF THE Difference between 50 and 150 of the Obiginaz. 

Alpha Scores. 


Orijr. Cor. 


Orig. Cor. 


Orig. CJor. 


Orig. Cor. Orig. Cor. 



In onr series as constmcted from the extensive data in 
Grades 5, 6, 7, 9, 12, and 13 the 170 to 190 difference is .3084. 
So in the terms of that series 190 to 194 = 8.39, 195 to 199 
= 12.98, and 200 to 204 = 15.82. 
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TABLE 64a. 

PSOTISZONAI. EQX7IVA1JCKTS TOS ARMT AXPHA BCOKES TROiS. 170 TO 209; SCAIiK 

AS IN Table 64. 


Oiig. 

Cor. 

Grig. 

Cor. 

Grig. 

Cor. 

Grig. 

Cor. 

170 

172.2 

180 

187.2 

190 

203 

200 

223 

171 

173.7 

181 

188.8 

191 

205 

201 

225 

172 

175.2 

182 

190.4 

192 

207 

202 

228 

173 

176.7 

183 

192. 

193 

209 

203 

231 

174 

178.2 

184 

193.6 

194 

211 

204 

234 

175 

179.7 

185 

195.2 

195 

213 

205 

237 

176 

181.2 

186 

196.7 

196 

215 

206 

240 

177 

182.7 

187 

198.3 

197 

217 

207 

243 

178 

184.2 

188 

199.8 

198 

219 

208 

246 

179 

185.7 

189 

201.4 

199 

221 

209 

249 


Averaging the two determinations for 190 to 194 (6.19 
and 8.39), we have 7.28 as the best estimate for that. We 
have 12.98 as the best estimate for 195 to 199, and 15.82 as 
the best estimate for 200 to 204. This would give us 20.26 
for the interval 190 to 199, with a probability that the in- 
terval from 200 to 209 would be still larger. 

We have not incorporated these determinations with 
the others, because they are less secure. There can be no 
doubt, however, that anyone will be much nearer the truth 
by using them than by using the original score values for 
Army Alpha. There can be little doubt also that the units 
of Alpha represent in general progressively greater incre- 
ments of ability from 130 to 210. Subject to further inves- 
tigations we offer Table 64a as a scale of provisional 
equivalents in truly equal units for Army Alpha scores of 
170 to 209. 

The values of the letter-grade intervals used in the 
Army reports are as follows, in equal units: 


Griginal 

209 to 135 

High End of A to High End of B- 

Corrected 

249 to 135 

134 to 

105 

Hi^^h £nd of B— to 

High End of C -H 

134 to 

105 

104 to 

75 

High End of C + to 

High End of C 

104 to 

76 

74 to 

45 

High End of C to 

High End of C — 

75 to 

45 

44 to 

25 

High End of C — to 

High End of P 

44 to 

21 

24 to 

15 

High End of P to 

High End of P — 

20 to 

t 

14 to 

0 

High End of P- to 

Liow End of P- 

f to 

t 
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THE NATIOHAI. INTEULIGENCE EXAMINATION A 

Consider next the case of the National Intelligence Ex- 
amination A. We use first the scores from 1,668 pupils in 
Grade 6 and 494 pupils in Grade 9. The results of the 
derivation of values in equal units are shown in Table 66. 
The procedure by which these are obtained is just the same 
as that shown in detail in the case of Army Alpha. The two 
sets of values are combined with weights of 3 and 1. 

TABLE 65. 

ARMT AXPHA DlSTJUBUTIOir OF SCOBSS OF 216 COLIJCOE Gbaouatbs. 


100 to 104 

105 to 109 

1 

110 to 114 

2 

115 to 119 

120 to 124 

2 

125 to 129 

3 

130 to 134 

5 

135 to 139 

1 

140 to 144 

4 

145 to 149 

6 

150 to 154 

8 

155 to 159 

9 

160 to 164 

6 

165 to 169 

15 

170 to 174 

24 

175 to 179 

22 

180 to 184 

38 

185 to 189 

22 

190 to 194 

20 

195 to 199 

18 

200 to 204 

8 

205 to 209 

2 


We next use scores from 1,679 pupils in Grade 7 and 482 
pupils in Grade 8, the results, weighted by 3 and 1, being 
as shown in Table 67. The values here are, in the last col- 
umn, divided by a factor such that the total difference 70 
to 170 is the same as for the determinations from Grades 
6 and 9. 




TABLE 66. 

National A: Grades 6 (n = 1668) and 9 (n=494). 
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National A: Grades 7 (n = 1679) and 8 (n = 482) 
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TABLE 68. 

National A. Summary or Determinations of Values in Equal Units. 
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7.37 7.78 7.58 150 “ 159 13.22 13.51 13.37 

8.20 9.20 8.70 

5.25 6.53 5.89 160 « 169 13.45 15.73 14.59 

5.26 9.97 7.62 



THE TBAHSF0BM:ATI01!r OF STAHDAKD SC70BES 


243 


In Table 68 the results of the 6, 9 and of the 7, 8 deter* 
zninations are pnt side by side and averaged, and measures 
of the unreliability of the averages are attached. 

We have made a provisional extension of the transmu* 
tations for National A down to original scores of 20 by 
using the assumption that Grades 5 and 4 will show ap- 
proximately normal distribution of perfectly measured in- 

TABLE 69. 

Nationai. A: Grade 4 (ii=1677) and Grade 5 (ii = 2487) 


Interval 

Original 

Distributiozis 

Values in Equal Units 

Aver- 

Average 


4 

5 

4 

5 

a|;e 

.924 

10- 19 

10 

3 





20- 29 

33 

7 

11.51 

9,30 

10.40 

9.61 

30- 39 

09 

23 

9.92 

10.19 

10.06 

9.30 

40- 49 

113 

63 

8.24 

10.44 

9.34 

8.63 

50- 59 

214 

118 

10.07 

9.42 

9.75 

9.01 

60- 69 

285 

275 

9.87 

12.00 

10.94 

10.02 

70- 79 

311 

347 

9.99 

10.02 

10,01 

9.25 

80- 89 

257 

417 

9.36 

10.13 

9.75 

9.01 

90- 99 

200 

412 

10.26 

10.01 

10.14 

9.37 

100-109 

106 

350 

9.49 

10.27 

9.88 

9.13 

110-119 

55 

226 

10.85 

9.55 

10.20 

9.42 

120-129 

7 

147 

7.69 

10.83 

9.26 

8.56 


2 

66 

10.59 

10.44 

10.52 

9.71 


22 

11 

1 

1 


tellect. Table 69 shows the original distributions, the values 
of the intervals in equal units by the assumption, the quo- 
tients when these are divided by 1/50 of the difference 
between 70 and 120 of the original scale, the averages for 
the two grades, and these averages after multiplication by a 
factor which equates the 70 to 120 difference with the 70 to 
120 difference of the scale derived by the use of Grades 6 
and 9. 
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TABIiB 70. 

Eqi7tvai;ents tob, Nahonaij A Scokes rsou 20 to 170, in a Bcaxa With 
Equal Units 1 = Approximately 1 /50 op the Difference Be- 
tween 100 AND 150 OF THE ORIGINAL SCALE. 


Orig, 

Oor. 

Orig. 

Cor. 

Orig. 

Cor. 

Orig. 

Cor. 

Orig. 

Cor. 

20 

30.2 

50 

57.8 

80 

82.4 

110 

110 

140 

139 

1 

31.2 

1 

58.6 

1 

83.2 

1 

111 

1 

140.1 

2 

32.2 

2 

59.4 

2 

84.1 

2 

112 

1 

141.2 

3 

33.2 

3 

60.2 

3 

85.0 

3 

113 

3 

142.3 

4 

34.1 

4 

61.0 

4 

85.8 

4 

114 

4 

143.4 

5 

35.1 

5 

61.8 

5 

86.6 

5 

115 

5 

144.5 

6 

36.1 

6 

62.6 

6 

87.4 

6 

116 

6 

145.6 

7 

37.0 

7 

63.4 

7 

88.2 

7 

117 

7 

146.7 

8 

38.0 

8 

64.2 

8 

89.0 

8 

118 

8 

147.8 

9 

38.9 

9 

65.0 

9 

89.8 

9 

119 

9 

148.9 

30 

39.8 

60 

65.7 

90 

90.7 

120 

120 

150 

150.0 

1 

40.8 

1 

66.5 

1 

91.6 

1 

120.9 

1 

151.3 

2 

41.8 

2 

67.3 

2 

92.6 

2 

121.8 

2 

152.6 

3 

42.7 

3 

68.1 

3 

93.6 

3 

122.7 

3 

154.0 

4 

43.6 

4 

68.9 

4 

94.6 

4 

123.6 

4 

155.3 

5 

44.6 

5 

69.7 

5 

95.6 

5 

124.5 

5 

156.6 

6 

45.5 

6 

70.5 

6 

96.5 

6 

125.4 

6 

158.0 

7 

46.4 

7 

71.3 

7 

97.5 

7 

126.7 

7 

159.3 

8 

47.3 

8 

72.1 

8 

98.5 

8 

127.2 

8 

161.6 

9 

48.2 

D 

73.0 

9 

99.5 

9 

128.1 

9 

162.0 

40 

49.1 

70 

73.8 

100 

100.5 

130 

129.0 

160 

163.4 

1 

50.0 

1 

74.6 

1 

101.4 

1 

130 

1 

164.8 

2 

50.9 

2 

75.5 

2 

102.4 

2 

131 

3 

166.2 

3 

51.8 

3 

76.3 

3 

103.3 

3 

132 

3 

167.6 

4 

52.7 

4 

77.2 

4 

104.3 

4 

333 

4 

169.0 

5 

53.6 

5 

78.0 

5 

105.2 

5 

134 

5 

170.5 

6 

54.5 

6 

78.9 

6 

106.2 

6 

135 

6 

172.0 

7 

55.3 

7 

79,8 

7 

107.1 

7 

136 

7 

173.5 

8 

56.2 

8 

80.6 

8 

108.1 

8 

137 

8 

175.0 

9 

57.0 

9 

81.5 

9 

109.0 

9 

138 

9 

176.5 









170 

178.0 


We combine these results with that from Grades 6, 7, 8, 
and 9, allowing equal weight to each of the two, and so have, 
as provisional values for these low intervals, the following : 


20-29 

9.61 

30-39 

9.30 

40-49 

8.63 

50-59 

7.92 

60-69 

8.12 
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Using these values np to 70 and that of Table 68 from 
70 on, and making the original scale and the scale in equal 
units coincide at 120, we have Table 70 as our transmuting 
table. 

THE OTIS ADVANCED EXAMINATION 

In the case of the Otis Advanced Examination we have 
the distributions shown in Table 71. We obtain the o values 
in equal units for each interval for each group, as shown in 
the case of Army Alpha. In Groups I, II, and III we then 
divide each of these by the difference between 70 and 140 
(in equal units) ; average I and II with respective weights 
of 2 and 1; combine this average with III, giving equal 
weight to Grade 6 and to Grade 9. In Group IV we divide 
each of the a values by the difference between 100 and 170 
(in equal units) and then multiply each by a factor which 
makes the 100 to 170 difference for Group IV equal to that 
for the I, IT, III weighted average. The I, II, III weighted 
average and the IV result are then averaged with weights 
of 3 and 1, respectively. The essential steps in these com- 
putations are shown in Table 72, the last column of which 
shows the combined estimate of the relative values of the 
10-point intervals from 30 to 200 in terms of equal units. 
For convenience in interpretation these values are divided 
by 1.06, which makes the unit of the equal-unit scale 1/100 
of the difference between 70 and 170 of the original scale. 
By interpolation and smoothing, letting the two scales coin- 
cide at 100, we obtain the equivalents of Table 73. It may 
be noted that what scant data we have above 200 indicate 
that the rise from 12.52 to 17.77 (or 11.81 and 16.76, after 
division by 1 .06) is not a matter of the sampling error. The 
data give 21.60 (or 20.38) as the value for 200 to 209. 

The interval from 20 to 29 has a value of 18.30 (17.26 
after division by 1.06) by the sixth-grade groups, but this 
is too unreliable for use without confirmation. We have 
considered the facts for a fifth-grade population of 3,058 
individuals and a fourth-grade population of 1,500 pupils. 
We do not, of course, know that in these grades the distri- 
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TABIiE 71. 

Otis Asvanois): Distributions. 


Interral 

I 

Grade 6 
n = 4298 

u 

Grade 6 
n = 1654 

m 

Grade 9 

R = 3627 

IV 

Grade 12 
n n 1226 

10 to 

19 

3 

1 



20 

29 

19 

12 



30 

39 

74 

23 



40 

49 

168 

56 



50 ‘‘ 

59 

334 

107 

9 


60 ‘‘ 

69 

504 

183 

40 


70 

79 

659 

243 

79 


80 

89 

738 

268 

174 

6 

90 

99 

587 

244 

262 

23 

100 

109 

499 

209 

443 

38 

no 

119 

346 

135 

520 

58 

120 

129 

193 

93 

541 

95 

130 

139 

97 

45 

547 

153 

140 

149 

51 

25 

409 

187 

150 

159 

20 

8 

317 

191 

160 

169 

5 

1 

190 

187 

170 

179 



62 

139 

180 '' 

189 

1 

1 

24 

85 

190 

199 



10 

50 

200 

209 




13 

210 

219 




1 


butions of truly measured intellect are of Form A ; but their 
low end will not diverge enough from the corresponding 
sections of Form A to invalidate the comparisons which we 
shall make. 

Assuming the low end to be of the geometrical form of 
the corresponding section of Form A, and expressing the 
true values of the interval 10 to 19, and of the interval 20 
to 29 in terms of the interval 30 to 39, we find the following : 

The relevant facts are that in Grade 4 we have 33 cases 
(out of a total of 1,500) from 0 to 9, 121 cases from 10 to 19, 
^6 cases from 20 to 29, and 248 cases from 30 to 39; in 
Grade 5 we have 6, 41, 112 and 257 (out of the total of 
3,058) in these same intervals. 

Interval Grade 4 Grade 5 Average 

10-19 1.65 1.37 1.51 

20-29 1.43 1.00 1.22 
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We allow equal weights in averaging, because the larger 
population of Grade 5 is olfset by the larger proportion of 
Grade 4 in the intervals studied. 

TABLE 72. 

Ons Advakced: EquivAiiENTS fq& Each 10-Poiht Inte&vai. of the OaioiHAii 

ScAiiE IN Eqcai. Units. 


luteryal 

A 

6 

n = 4298 

B 

6 

n = 1654 

c 

2A + B 

3 n 

D 

9 

= 3627 

E 

o+i> 

2 

P 

12 

n = 1226 

G 

3E+ P 
~4 

H 

G/1.06 

30 to 39 

14.29 

10.04 

12.86 


12.86 


12.86 

12.1 

40 to 49 

11.89 

10.80 

11.53 


11.53 


11.53 

10.9 

50 to 59 

11.74 

10.54 

11.34 


11.34 


11.34 

10.7 

60 to 69 

10.91 

11.18 

11.00 


11.00 


11.00 

10.4 

70 to 79 

10.89 

10.80 

10.86 

10.13 

10.50 


10.50 

9.9 

80 to 89 

10.99 

10.46 

10.81 

10.69 

10.75 


10,75 

10.1 

90 to 99 

9.47 

9.84 

9.60 

9.35 

9.48 


9.48 

8.9 

100 to 109 

10,00 

10.31 

10.10 

10.74 

10.42 

10.52 

10.45 

9.9 

110 to 119 

10.23 

9.18 

9.88 

9.90 

9.89 

9.10 

10.09 

9.5 

120 to 129 

9.63 

10.45 

9.87 

9.56 

9.72 

9.74 

9.72 

9.2 

130 to 139 

11.54 

9.38 

10.82 

10.41 

10.62 

11.27 

10.78 

10.2 

140 to 149 




9.77 

9.77 

11.24 

10.14 

9.6 

150 to 159 




11.23 

11.23 

10.96 

11.15 

10.5 

160 to 169 




13.10 

13.10 

12.08 

12.84 

12.1 

170 to 179 




10.16 

10.16 

12.27 

10.69 

10.1 

180 to 189 






12.52 

12.52 

11,8 

190 to 199 






17-77 

17.77 

16.8 


Multiplying the 12.86 of Column G and the 12.1 (more 
exactly 12.13) of Column H of Table 72 by 1.51 and 1.22, we 
have these values for the intervals : 

10-19 19.42 (or 18.32 when divided by 1.06). 

20-29 15.69 (or 14.80 when divided by 1 . 06 ). 

We may use these as provisional values subject to further 
investigation. They are used in the extension of Table 73 
by Table 73a. 

THE HAGGEBTY EXAMINATION, DELTA 2 

In the case of the Haggerty Delta 2 we have the distribu- 
tions shown in Table 74. After estimating the frequencies 
18 
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TABLE 73. 

EQl7IVAI<ENTrs FOR OTIS ADVANCED SCORES FROM 30 TO 200 IN A SCALE WITH 


Equal Units. 1 = the Difference between 50 

AND 170 OF THE ORIOINAL SCORES. 


o 

c 

O 

C 

O 

c 

o 

C 

O 

c 

30 

26.4 

70 

70.6 

110 

109.5 

150 

147.8 

190 

192.3 

31 

27.7 

71 

71.6 

111 

110.5 

151 

148.9 

191 

193.6 

32 

28.9 

72 

72.6 

112 

111.4 

152 

149.9 

192 

195.1 

33 

30.2 

73 

73.6 

113 

112.4 

153 

151.0 

193 

196.6 

34 

31.4 

74 

74.6 

114 

113.3 

154 

152.0 

194 

198.1 

35 

32.6 

75 

75.6 

115 

114.3 

155 

152.9 

195 

199.7 

86 

33.8 

76 

76.6 

116 

115.2 

156 

154.0 

196 

201.3 

37 

35.0 

77 

77.6 

117 

116.2 

157 

155.0 

197 

203.0 

38 

36.2 

78 

78.6 

118 

117.1 

158 

156.1 

198 

204.7 

39 

37.4 

79 

79.6 

119 

118.1 

159 

157.2 

199 

206.5 

40 

38.6 

80 

80.6 

120 

119.0 

160 

158.3 

200 

208.3 

41 

39.7 

81 

81.6 

121 

120.0 

161 

159.4 



42 

40.8 

82 

82.5 

122 

120.9 

162 

160.5 



43 

41.9 

83 

83.5 

123 

121.9 

163 

161.6 



44 

43.0 

84 

84.5 

124 

122.8 

164 

162.7 



45 

44.1 

85 

85.4 

125 

123.8 

165 

163.8 



46 

45.2 

86 

86.4 

126 

124.7 

166 

164.9 



47 

46.3 

87 

87.4 

127 

125.7 

167 

166.0 



48 

47.4 

88 

88.3 

128 

126.6 

168 

167.1 



49 

48.4 

89 

89.3 

129 

127.6 

169 

168.2 



50 

49.5 

90 

90.3 

130 

128.6 

170 

169.3 



51 

50.6 

91 

91.3 

131 

129.5 

171 

170.4 



52 

51.6 

92 

92.2 

132 

130.5 

172 

171.5 



53 

52.7 

93 

93.2 

133 

131.4 

173 

172.6 



54 

53.8 

04 

94.2 

134 

132.4 

174 

173.7 



55 

54.9 

95 

95.1 

135 

133.4 

175 

174.8 



56 

56.0 

96 

96.1 

136 

134.3 

176 

175.9 



57 

57.0 

97 

97.1 

137 

135.3 

177 

177.0 



58 

58.1 

98 

98.0 

138 

136.2 

178 

178.1 



59 

59.2 

99 

99.0 

139 

137.2 

179 

179.3 



60 

60.2 

100 

100.0 

140 

138.2 

180 

180.5 



61 

61.3 

101 

101.0 

141 

139.1 

181 

181.6 



62 

62.3 

102 

101.9 

142 

140.1 

182 

182.8 



63 

63.4 

103 

102.9 

143 

141.0 

183 

183.9 



64 

64.4 

104 

103.9 

144 

142.0 

184 

185.1 



65 

65.5 

105 

104.8 

145 

143.0 

185 

186.3 



66 

66.5 

106 

105.7 

146 

143.9 

186 

187.5 



67 

67.5 

107 

106.7 

147 

144.9 

187 

188.7 



68 

68.6 

108 

107.6 

148 

145.8 

188 

189.9 



69 

69.6 

109 : 

108.6 

149 

146.8 

189 

191.1 
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TABUE: 73a. 


Pbotisjonaxi Values for Otis Advakced Scores from 10 to 8d. 


o 

C 

O 

O 

O 

C 

O 

0 

10 

- 6.7 

15 

2.6 

20 

11.6 

25 

19.1 

11 

- 4.8 

16 

4.4 

21 

13.1 

26 

20.6 

12 

- 3,0 

17 

6'.2 

22 

14.6 

27 

22.1 

13 

- 1.1 

18 

8.0 

23 

16.1 

28 

23.6 

14 

.8 

19 

9.8 

24 

17.6 

29 

25 


in intervals of 10 from the irregular arrangement of III 
and IV, the values of each interval of each group in equal 
units are computed. These values for I, II, and III are put 
in terms of the difference between original 70 and original 
130, to make them comparable. The two Grade 9 deter- 
minations are then combined with weights of 1 and 3, re- 
spectively. With these averages are combined the deter- 
minations from Grade 6, with weights of 2 for the former 
and 1 for the latter. The determinations from Grade 12 are 


TABLE 74. 

Haggertt Delta 2 Distributions 


Interval 

1 

Grade 6 
n = 916 

II 

Grade 9 
n = 473 

III 

Grade 9 
n = 1995 

rv 

Grade 12 

11 = 668 

10 to 19 

1 





20 



25 to 42 

1 


SO 

4 





40 

12 


43 to 54 

3 


50 

39 

1 

55 to 65 

10 


60 

87 

5 

66 to 76 

29 


70 

127 

6 




80 

161 

36 

77 to 86 

73 

1 

90 

164 

54 

87 to 99 

225 

13 

100 

154 

89 

100 to 114 

555 

45 

110 

86 

109 

115 to 119 

212 

36 

120 

61 

79 

120 to 129 

415 

121 

130 

17 

73 

130 to 139 

283 

162 

140 

2 

42 

140 to 149 

155 

170 

150 to 159 

1 

12 

150 to 159 

31 

102 

160 to 169 


1 

160 to 169 

3 

16 




170 to 179 


2 
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made comparable with this composite determination by 
multiplying them by a factor such as makes the 100 to 150 
difference the same for the Grade 12 group as for the com- 
posite. They are then combined with the composite deter- 
mination, the weights being 1 for the Grade 12 items and 3 
for the composite. The essentials of these computations 
appear in Table 75, the last column of which gives the final 
estimate. The units of the Haggerty Delta 2 score become 
progressively “harder’^ (that is, larger) when put in equal 
units, from some point in the 70 ’s up to 160. 

TABLE 75. 

Haooertt Delta 2 : Values in Equal Units. 


Interval n 

A 

= 916 

B 

n = 473 

C 

n = 1995 

D 

B + 3C 

4 

A + 2D 

3 

F 

n=668 

O 

3B + P 
4 

50 to 59 

10.85 




10.85 


10.85 

60 to 69 

10.55 




10.55 


10.55 

70 

9.31 

5.20 

9.29 

8,27 

8.62 


8.62 

80 

9.19 

13.25 

8.38 

9.62 

9.48 


9.48 

90 

9.03 

9.62 

9.08 

9.21 

9.15 


9.15 

100 

10.30 

10.42 

11.06 

10.90 

10.70 

10.05 

10.54 

110 

8.78 

11,48 

10.67 

10.87 

10.17 

9.50 

10.01 

120 

13.41 

9.82 

11.53 

11.10 

11.87 

12.61 

12.06 

130 


16.61 

11.87 

13.06 

13.06 

12.86 

13.01 

140 


10.37 

16.16 

14.71 

14.71 

15.46 

19.86 

150 






20.88 

20.68 


Interpolating, smoothing, expressing the values in 
terms of 1/60 of the difference between original 60 and orig- 
inal 120, and letting the two series coincide at 90, we have 
the equivalents of Table 76. 

THE TEBMAN GROUP TEST 

In the case of the Terman Group Test of Mental Ability, 
we have the scores of 5,582 pupils in Grade 7, 9,087 in 
Grade 8, 10,881 in Grade 9, 6,730 in Grade 10, 4,206 in 
Grade 11, and 4,886 in Grade 12. [Terman, ^23, p. 9.] 
These are reported in the form of the point on the scale 
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TABLE 76. 

EqdivaiiEmts fos Hagoebtt Delta 2 Scores from: 50 to 160, at a ScaxiE 

WITH Equal Units. 


o 

c 

-o 

o 

O 

C 

50 

50.6 

87 

87.1 

124 

124.8 

51 

51.7 

88 

88.1 

125 

126.0 

52 

52.8 

89 

89.0 

126 

127.2 

53 

53.9 

90 

90.0 

127 

128.4 

54 

55.0 

91 

90.9 

128 

129.6 

55 

56.1 

92 

91.8 

129 

130.8 

56 

57.2 

93 

92.7 

130 

132.0 

57 

58.3 

94 

93.7 

131 

133.3 

58 

59.4 

95 

94.6 

132 

134.6 

59 

60.5 

96 

95.6 

133 

135.9 

60 

61.5 

97 

96.5 

134 

137.2 

61 

62.6 

98 

97.4 

135 

138.5 

62 

63.7 

99 

98.3 

136 

139 .S 

63 

64.7 

100 

99.3 

137 

141.1 

64 

65.8 

101 

100.3 

138 

142.4 

65 

66.9 

102 

101.4 

139 

143.7 

66 

67.9 

103 

102.4 

140 

145.0 

67 

69.0 

104 

103.4 

141 

14 G .9 

68 

70.0 

105 

104.5 

142 

148.8 

09 

71.0 

106 

105.5 

143 

150.7 

70 

72.0 

107 

106.5 

144 

152.6 

71 

72.9 

108 

107.6 

145 

154.5 

72 

73.8 

109 

108.6 

146 

156.4 

73 

74.7 

110 

109.7 

147 

158.3 

74 

75.5 

111 

110.7 

148 

160.2 

75 

76.4 

112 

111.7 

149 

162.1 

76 

77.2 

113 

112.8 

150 

164.0 

77 

78.1 

114 

113.8 

151 

166.1 

78 

79.0 

115 

114.8 

152 

168.2 

79 

79.8 

116 

115.9 

153 

170.3 

80 

80.7 

117 

116.9 

154 

172.4 

81 

81.6 

118 

117.9 

155 

174.5 

82 

82.5 

119 

119.0 

156 

176.6 

83 

83.4 

120 

120.0 

157 

178,7 

84 

84.3 

121 

121.2 

158 

180.8 

85 

85.2 

122 

122.4 

159 

183.0 

86 

86.2 

123 

123.6 

160 

185.0 



TABLE 77. 
Teeuan Geoup Test. 
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122 148- 164 177 189 194 12.11 12.27 12.49 12.52 13.31 13.12 

134 159 172 185 196 200 10.50 10.64 10.83 10.86 11.54 11.38 

147- 170 181 194 203 207 12.21 12.38 12.59 12.63 13.42 13.23 
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below which a certain permille of the group lies. Thus 
Table 77 states that 10 permille or 1 percent of the popula- 
tion of Grade 7 had scores up to 20, 25 permille had scores 
up to 25, 50 permille had scores up to 31, and so on. 

^ TABLE 78. 

Terman Gbouf Test. 

Sample of the Six Sete of Values in £qual Units Whence the General 
Transmutation Table Is Derived. 



Or. 7 

Gr. 8 

Gr. 9 

Gr. 10 

Gr. 11 

Gt. 12 

Average 

100 

100.00 

100.00 

100.00 

100.00 

100.00 

100.00 

100. 

101 

100.91 

100.92 

100.99 

101.04 

101-19 

100.98 

101. 

102 

101.82 

101.84 

101.98 

102.08 

102.38 

101.97 

102. 

103 

102.72 

102.76 

102.97 

103.11 

103.57 

102.95 

103. 

104 

103.63 

103.67 

103.97 

104.04 

104.76 

103.93 

104. 

105 

104.54 

104.58 

104.94 

104.98 

105.95 

104.92 

105. 

106 

105.45 

105.50 

105.90 

105.91 

106.82 

105.90 

106. 

107 

106.36 

106.41 

106.87 

106.85 

107.70 

106.88 

107. 

108 

107.26 

107.42 

107.83 

107.78 

108.57 

107.87 

108. 

109 

108.17 

108.44 

108.79 

108.71 

109.45 

108.85 

109. 

110 

109.10 

109.45 

109.76 

109.65 

110.32 

111.19 

110. 

111 

110.03 

110.46 

110.73 

110.58 

111.19 

113.54 

111. 

112 

110.96 

111.48 

111.70 

111.52 

112.07 

115.88 

112. 

113 

111.90 

112.42 

112.67 

112.45 

112.99 

116.48 

113. 

114 

112.83 

113.36 

103.60 

113.42 

113.90 

117.09 

114. 

115 

113,76 

114.30 

104.53 

114.39 

114.82 

117.69 

115. 

190 




195.75 

198.69 

199.79 

198. 

191 




197.15 

200.34 

201.25 

200. 

192 




198.56 

201.99 

202.70 

201. 

193 




199.96 

203.64 

204.16 

203. 

194 




201.36 

205.29 

205.62 

204. 

195 





206.94 

207.52 

207. 

196 





208.58 

209.41 

209. 

197 





210.50 

211.31 

211. 

198 





212.42 

213.21 

213. 

199 





214.34 

215.10 

215. 

200 





216 26 

217.00 

217. 


We therefore find the value in equal units for the given 
intervals, rather than for successive intervals of 5 or 10. 
These values are then divided by : 
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1/86 of the difference between 61 and 147 in the case of 
Grade 7. 

1/84 of the difference between 64 and 148 in the case of 
Grade 8. 

1/88 of the difference between 63 and 151 in the case of 
Grade 9. 

1/80 of the difference between 67 and 147 in the case of 
Grade 10. 

1/81 of the difference between 66 and 147 in the case of 
Grade 11. 

1/84 of the difference between 63 and 147 in the case of 
Grade 12. 

The results appear as the last six columns of Table 77. 
We then construct a transmutation table of values for each 
grade, in each case coinciding with the original scale at 100. 
A sample piece of this table is shown as Table 78. 

A final transmutation table (Table 79) is obtained by 
averaging all six values for each point of the original scale, 
or all five values, if there are only five, or all four or three, 
if there are onlv four or three. The table does not extend 
down or up beyond the points where three grade distribu- 
tions were available. 

THE MYERS MENTAL. MEASURE 

Using the scores of 724 pupils in Grade 6 and 311 pupils 
in Grade 9 in the Myers Mental Measure, we find the facts 
of Table 80 and construct Table 81 as the transmuting table. 
The two scales coincide at 46. 

THE PIXTNER NON-LANGUAGE TEST 

For the Pintner Non-Language Mental Test we have 
the scores of 1,237 pupils in Grade 6 and 258 in Grade 9 
shown in Table 82 (columns I and II). Prom these we de- 
rive the values in true units for each twenty-point interval 
and express them in terms of one two-hundredth of the dif- 
ference between 260 and 460. They appear as colunms III 
and IV of Table 82. When they are averaged, with weights 
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TABLE 79. 

TEBMAiT QaoTJP Test of Mentazj Abiutt: Vai.ues in Equaij Units of Each 
Point on the Orioinaii Spate fbou 35 to 193. 


o 

c 

o 

c 

o 

c 

o 

c 

o 

c 

35 

21 

70 

68 

100 

100 

130 

129 

160 

158 

6 

22 

1 

69 

1 

101 

1 

129 

1 

159 

7 

24 

2 

71 

2 

102 

2 

130 

2 

160 

8 

25 

3 

72 

3 

103 

3 

131 

3 

161 

9 

27 

4 

73 

4 

104 

4 

132 

4 

162 

40 

28 

75 

74 

105 

105 

135 

133 

165 

163 

1 

30 

6 

75 

6 

106 

6 

134 

6 

165 

2 

31 

7 

76 

7 

107 

7 

135 

7 

166 

3 

33 

8 

77 

8 

108 

8 

136 

8 

167 

4 

34 

9 

78 

9 

109 

9 

137 

9 

168 

45 

36 

80 

80 

110 

110 

140 

138 

170 

170 

6 

37 

1 

81 

1 

111 

1 

139 

1 

171 

7 

39 

2 

82 

2 

112 

2 

140 

2 

172 

8 

40 

3 

83 

3 

113 

3 

341 

3 

174 

9 

41 

4 

84 

4 

114 

4 

142 

4 

176 

50 

43 

85 

85 

115 

115 

145 

143 

175 

176 

1 

44 

6 

86 

6 

116 

6 

144 

G 

178 

2 

46 

7 

87 

7 

117 

7 

145 

7 

179 

3 

47 

8 

88 

8 

118 

8 

146 

8 

181 

4 

49 

9 

89 

9 

118 

9 

147 

9 

152 

55 

50 

90 

90 

120 

119 

150 

148 

180 

183 

6 

51 

1 

91 

1 

120 

1 

149 

1 

185 

7 

52 

2 

92 

2 

121 

o 

150 

2 

186 

8 

54 

3 

93 

3 

122 

3 

151 

3 

188 

9 

55 

4 

94 

4 

123 

4 

152 

4 

189 

60 

56 

95 

95 

125 

124 

155 

153 

185 

191 

1 

58 

6 

96 

6 

125 

6 

154 

6 

192 

o 

59 

7 

97 

7 

126 

7 

155 

7 

194 

3 

60 

8 

98 

8 

127 

8 

156 

8 

195 

4 

61 

9 

99 

9 

128 

9 

157 

9 

197 

65 

63 







190 

198 

6 

64 







1 

200 

7 

65 







2 

201 

8 

66 







3 

203 

9 

67 







4 

204 
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TABLJB 80. 

Mtebs Mentai. Measthoc; Grade 6 (n = 724) and Geiade 9 (n = 311); Valitbs 
OF Intebtads in Terms of Equal Units, Expressed as Multiples 


OP ^ X (Difference Between 36 and 81). 


Interval 

Grade 6 

Grade 9 

Average 

21 to 25 

9.69 


9.69 

26 “ 30 

6.78 


6.78 

31 36 

7.89 

11.83 

9.86 

36 40 

6.98 

6.10 

6.54 

41 45 

5.44 

7.36 

6.40 

46 '' 50 

5.89 

4.95 

5.42 

51 << 55 

5.01 

5.23 

5.12 

56 60 

6.43 

5.38 

5.91 

61 “ 65 

3.19 

3.46 

3.33 

66 70 

4.57 

4.39 

4.48 

71 75 

3.86 

5.62 

4.74 

76 80 

3.61 

1.62 

2.62 

81 85 


5.35 

5.35 


TABLE 81. 


Equivalents of Scores 

FROM 21 TO 

86 FOB Mvebs 

Mbntaii Mbasubb: 


In 'Equal 

Units. 


o c o 

c 

o c 

O C 


21 

5.7 

41 

39.6 

61 

61.0 

81 

78.0 

22 

7.7 

42 

40.9 

62 

62.0 

82 

79.0 

23 

9.7 

43 

42.2 

63 

62.9 

83 

80.0 

24 

11.6 

44 

43.4 

64 

63.9 

84 

81.0 

25 

13.5 

45 

44.7 

65 

64.8 

85 

82.0 

26 

15.4 

46 

46.0 

66 

65.7 

86 

82.9 

27 

16.7 

47 

47.0 

67 

66.7 



28 

18.1 

48 

48.1 

68 

67.6 



29 

19.4 

49 

49.2 

69 

68.5 



30 

20.8 

50 

50.3 

70 

69.4 



31 

22.2 

51 

51.4 

71 

70.3 



32 

24.2 

52 

52.4 

72 

71.3 



33 

26.2 

53 

53.4 

73 

72.2 



34 

28.2 

54 

54.4 

74 

73.1 



35 

30.2 

55 

55.5 

75 

74.0 



36 

32.1 

56 

56.5 

76 

74.9 



37 

34.4 

57 

57.4 

77 

75.5 



38 

35.7 

58 

58.3 

78 

76.1 



39 

37.0 

59 

59.2 

79 

76.7 



40 

38.3 

60 

60.1 

80 

77.3 
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of 4 and 1, we have column V. The transmutation table, 
which coincides with the ori^nal scale at 300, appears as 
Table 83. 

THE I. E. B. TEST OF SELECTIVE AND BELATIONAL THINKING, 
GENEBALIZATION AND OBGANIZATION 

For the I. E. R. Test of Selective and Relational Think- 
ing, Generalization and Organization, we have scores from 


TABLE 82. 

PiNTNER Noir-LANODAGE MENTAX TEST. OBIGINAli SCORES AND VALUES 
OP Intervals dst Equal Units. 


Interval 

I 

Original 
Grade 6 
n = 1237 

II 

Scores 
Grade 9 
11=: 258 

III IV 

Values of Intervals 
in £qual Units 
Grade 6 Grade 9 

V 

Weighted 

Average 

Value 

0 

to 

19 

1 





20 

i i 

39 






40 

€ € 

59 

2 





60 

tf 

79 

2 





80 

( ( 

99 

6 





100 

f t 

119 

11 


19.6 


19.6 

120 

i < 

139 

17 


22.0 


22.0 

140 

< i 

159 

32 


13.5 


13.5 

160 

t i 

179 

28 

1 

13.7 


13.7 

180 

C t 

199 

37 

1 

12.3 


12.3 

200 

i t 

219 

41 


12.2 


12.2 

220 

i t 

239 

48 

I 

12.2 


12.2 

240 

c < 

259 

56 

5 

19.3 

29.7 

21.4 

260 

< t 

279 

101 

6 

15.5 

19.7 

16-3 

280 

1 1 

299 

92 

13 

19.8 

26.4 

21.1 

300 

* c 

319 

125 

13 

18.6 

18.2 

18.5 

320 

< 1 

329 

119 

11 

17.7 

12.7 

16.7 

340 

i < 

359 

108 

27 

23.4 

25.3 

23.8 

360 

i 4 

379 

126 

21 

19.3 

16.8 

18.8 

380 

i i 

399 

85 

28 

23.2 

21.1 

22.8 

400 

i * 

419 

78 

26 

17.4 

19.4 

17.8 

420 

4 < 

439 

42 

25 

23.9 

19.8 

23.1 

440 

< < 

459 

38 

23 

20.9 

20.7 

20.9 

460 

1 1 

479 

20 

17 


18.7 

18.7 

480 

ti 

499 

10 

14 


19.8 

19.8 

500 

li 

519 

7 

14 


30.3 

30.3 

520 

1 1 

539 

3 

6 


23.0 

23.0 

540 

4 4 

559 

1 

2 




560 

4 4 

579 


3 




580 



1 

1 





FOE PlNTNKR NOK-LANGUAOE SCOEES FROM 100 TO 880 , IN A SCALK WITH EqUAIi XJNITS. i 
ORiGiNAii Scores and the Scores Transmuted Into a Scaue With EquAD Units. 


THE MEASUEEMENT OF IKTELLIGENCE 


f- 4 rHC<ICQ‘^<»«<Ot^OOOiOiHe 3 eOTjilO?Ot>-t^OOO> 0 ©^W'^U 5 ^t*«OiHeieO^»OjeOOOSOi-H 

eocQcocoeocQeococQcocccQcocOffOcocoeococococQcococococoeocQCoeocoeocoeocQCOoocoeT) 


OrHC 9 M'^tA«Ot^COO»OiH 03 CO<^irdCOb« 000 )OrHC^eO'^lACOt«XO>Or-ie 9 CO^Ud^fr*QOA 
^ O «0 b* 

CO CO CO CQ 


©iH©acOCOM<» 0 «Ot^OOQOOiOrHwCOcO‘^» 0 <Ot»OOOsOC^CO'TH» 0 ^t'OiOrHCleO» 0 <O^OOe> 

OOOOOOOOOOOOT-IrHrHiHr-lTHr-lr-lTHr-li-mNOIOJWOlWMOacoeOCOCOeocOCOcoeO 

COCOeOCQCOCOCOCOCOCOCOCOCOCOCOCOCOCOCOCOCOCOCOCOCOCOCOCOCOCOCQCOCOCOCOCOCOeOCOCO 


OTHCaCO'^U 3 ©b*aO©OrHe 9 CO^krd©t«' 00 ©OrHe 4 eO'« 41 lA©b*QO©OiHeQeO^IO©b* 000 ) 
O r-l W CO 

CO CO CO CO 


COCO'^W©t^QOOO©OrHOqcqcO’^lrt©C^*^ 00 ©OrHe;iffO'^IO©h-OOOirHNCO' 4 *W 5 ©^OOOS 

©©©©©©©©©»^l>>t«.t^fc^t^l»t^t«twb-t>QOOOQOXXOOOOOOODaC©©© 0 )© 0 >©©© 

OaCK]CQC^(MC 4 C>IC^O)C 4 CaC 4090 ^C 4 C 40 ) 04 GqcaC 4 C^C<l 04 C^GgcaC<l 04 Ca 04040401 &QC^] 09 e 904 fia 


OiHeNlC 0 '^ti 0 ©t^ 00 ©Oi- 1 C 4 C 0 ^m©t« 00 ©OrHC 4 C 0 '^l 0 ©b' 00 ©Or-lC 4 C 0 '^U 3 ©b>a 0 a 
© I- 00 © 

C 4 03 ca 09 


0 >OOr-liH(MC 0 C 0 '^JO»O©©l >-00 00 ©©OTHrHC 0 C 0 '^©Wa 0 ©Oi^C 9 C 0 '^M 0 ©^- 00 ©THW 

CqcoCOCOcOCOCOCOCOcOCOCOCOeOCOCOCOCO'^'^'^'^TH 4 l-^^^'^<lO »0 1 ft»f 5 W»OWOW 5 kOW©© 
(M03CqO]CM03<MCsJOl09C^(M03CqC4(M01C3Cq<MOIO)01C3030)0309CqCs|010IOI0401090l010IOI 


Oi-ltMc 0 THO©t^ 0005 Oi-l 03 C 0 '^l 0 ©t>' 00 ©OrH 03 CQTj 1 ir 9 ©l'-Q 0 ©OrH 0 QC 0 ^«D©l> 00 © 
01 CO IfS 

O] 03 cq O] 


IO» 0 ©Ob»OOQO©OJOr><r-lC 3 COCO'-f'^iO©©t^r^OO©©OiHTHCl<MCO'^<^» 0 » 0 ©t^t»OOOD 

OOOOOOOOOi-liHfHiHr-lT-lTHiHTHp-lTHrHTHr-tr-lrH(ncqCq<MO>lCqOIC<IC3<MCq030309Cvl 

01C30)Cqcq03!M04C3Cq0304C303(MC9CqCMCqC30109CvlcaOIO)(MOslCqOIOiCS]090409CV|OI030IC3 


Ol-i 01 CO'!j<lO©l>OD©Oi-IC<|CO’^IO©t^aO©Or-IOlCO'^IO©l^aD©Or-lCslCO^»f 3 ©l>QO© 
00 © O iH 

ri r-l 03 03 


r^Q 0 ©©OTHrHC 3 e 0 e 0 ^l 0 l 0 ©r>t- 00 ©©Or-lC 30 JC 0 -»t<^O©©l>-aD 00 ©OOrH 0309 C 0 T!< 
t>*t^l^t^a 0 00 Q 0 Q 0 a 00000 Q 000 a 0 00 00 X 00 »©© 05 © 04 © 0 i 0 i 0 i©© 0 J©©OOOOOOO 


OiH 03 CO^IO©l>aD©Or-IC 3 CO^IQ©^*ao©Or-lC 3 COr^lO©t^X©Or-l 03 CO^) 0 ©t^aO© 
I lO © 


©t^aOOaOiH 03 COTttU 3 ©t^OOOD©OrHOICO'^ 0 © 00 ©OiHC 3 CO'^IO©t^©OTHNeOrj<IO© 


. pr-IQqeO'^ia©t>QO©Or-iC 3 CO^lOOfc«X©OrH 03 cO'«^ 0 ©t*OOOaOiH 01 CO^IO©fc^» 0 » 

O o t-i C 3 eo 


THE TBAHSrOBMlATIOH OF STAFTDABD SOOBBS 


259 


3 231 pupils in Grade 9 and from two groups (of 1,666 and 
972) in Grade 12, as shown in Table 84. The values 
point intervals in true units derived from facts of Table 84 
appear in Table 85. Table 86 is the transmutation table 


, TABLE 84. 

IB.B. T^ts of Sraornn. .Ki. Thwhmo. .ni. 

Oeoanization; Distribtjtions in Grade 9 and Grade 1-2- 



Grade i 
n = 3231 

40 to 49 

2 

50 ** 59 

6 

60 ** 69 

15 

70 “ 79 

15 

80 “ 89 

32 

90 99 

66 

100 109 

82 

110 119 

127 

120 “ 129 

168 

130 “ 139 

200 

140 “ 149 

271 

150 “ 159 

258 

160 169 

295 

170 “ 179 

272 

180 ** 189 

303 

190 “ 199 

246 

200 “ 209 

228 

210 “ 219 

195 

220 “ 229 

143 

230 “ 239 

115 

240 “ 249 

74 

250 “ 259 

49 

260 " 269 

26 

270 “ 279 

20 

280 “ 289 

8 

290 “ 299 

5 

300 “ 309 

4 

310 319 

4 

320 “ 329 

330 “ 339 

1 

340 349 

350 “ 359 

360 ** 369 

370 “ 379 

1 


Grade 12 
n := 1666 n = 972 


1 


1 

1 

6 

7 

11 

20 

32 

39 

56 

79 

94 

119 

119 

129 

142 

145 

134 

145 

103 

77 

54 

47 

41 

30 

12 

8 

10 

4 


1 

3 

4 
13 
20 
20 
37 
31 
51 

65 
76 
76 
95 

71 
86 

66 

72 
59 
32 
36 
30 

9 

6 

5 

5 

1 


1 1 
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without smoothing. For rough work, it is sufficient to take 
the original scores at their face value. They are very, very 
close to the true values from 130 to 250, and are nowhere 
more than 3.7 points off. 


TABLE 85. 

I.E.B. Test or Selective and Relational Thinking, Generalization ahd 
Obganization. Values or IntesvaijS or Original 
Scale in Equal Units. 


Interval 

Orade 

9 

n = 3231 

Grade 

12 

n=1666 

Grade 

12 

n=:972 

Grade 

12 

Average 

Grades 

9 and 12 
Average 

90 to 

99 

11.9 




11.9 

100 

109 

9.7 




9.7 

110 ** 

119 

10.6 




10.6 

120 

129 

10.4 




10.4 

130 

139 

9.8 

10.4 

13.5 

11.9 

10.9 

140 

149 

11.1 

10.7 

8.8 

9.75 

10.4 

150 

159 

9.3 

9.1 

11.4 

10.25 

9.8 

160 

169 

10.0 

9.9 

7.4 

8.65 

9.3 

170 

179 

9.0 

10.4 

9.7 

10.05 

9.6 

180 

189 

10.5 

9.9 

10.3 

10.1 

10.3 

190 "" 

199 

9.3 

10.5 

10.3 

10.4 

9.9 

200 ** 

209 

10.0 

9.4 

9.4 

9.4 

9.7 

210 

219 

10.3 

9.3 

11.2 

10.25 

10.3 

220 “ 

229 

9.7 

9.9 

8.3 

9.1 

9.4 

230 

239 

10.9 

10.2 

10.7 

10,45 

10.7 

240 

249 

10.3 

10.1 

8.9 

9.5 

9.9 

250 

259 


12.1 

11.2 

11.65 

11.7 

260 “ 

269 


10.3 

11.5 

10.9 

10.9 

270 

279 


9.3 

7.9 

8.6 

8.6 

280 “ 

289 


8.0 

11.8 

9.9 

9.9 

290 ** 

299 


9.0 

16.1 

12.55 

12.6 

300 " 

309 


11.1 

7.7 

9.4 

9.4 

310 

319 


12.6 

7.2 

9.9 

9.9 


THE BROWN UNIVERSITY PSYCHOLOGICAL EXAMINATION 

The results of similar computations for the Brown Uni- 
versity Psychological Fxamination appear in Tables 87 
and 88. 



TABLE 86. 

Traksforuation Table. I.E.E. Tests or Selective and Eelational Thinking, Generalization and Organization. 
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TABLE 87, 

BaowK* University PsYCHOiiOGiCAL Examination. Grades 12, N = 3333 and 

13, N = 2118. 


A 

Value of interval Values in terms of 


Interval 

in equal units 

Grade 12 Grade 13 
n = 3333 nr: 2118 

1/35 of the difference 
between 35 and 70 
Grade 12 Grade 13 

Average 

20 to 

24 

.5304 


6.25 


6.25 

25 

29 

.5800 


6.84 


6.84 

30 

34 

.4764 


5.62 


5.62 

35 

39 

.4500 

.4260 

5.30 

5.13 

5.22 

40 

44 

.3780 

.3741 

4.46 

4.50 

4.48 

45 

49 

.4172 

.3442 

4.92 

4.14 

4.53 

50 

54 

.3976 

.3851 

4.69 

4.63 

4.66 

55 

59 

.4179 

.4500 

4.93 

5.42 

5.18 

60 

64 

.4263 

.4495 

5.02 

5.41 

5.22 

65 “ 

69 

.4805 

.4798 

5.67 

5.77 

5.72 

70 '' 

74 


.4908 


5.91 

5.91 

75 

79 


.6048 


7.28 

7.28 


TABLE 88. 


Equivalents of Soores from 20 to 80 for the Brown University 
Psychological Examination, in Equal Units. 


O 

C 

O 

C 

O 

C 

20 

17.0 

40 

41.0 

60 

59.8 

1 

18.2 

1 

41.9 

1 

60.8 

2 

19.5 

2 

42.8 

2 

61.9 

3 

20.7 

3 

43.7 

3 

63.0 

4 

22.0 

4 

44.6 

4 

64.0 

25 

23.4 

45 

45.5 

65 

65.1 

6 

24.6 

6 

46.4 

6 

66.2 

7 

26.0 

7 

47.3 

7 

67.4 

8 

27.4 

8 

48.2 

8 

68.5 

9 

28.8 

9 

49.1 

9 

69.6 

30 

30.2 

50 

50 

70 

70.8 

1 

31.3 

1 

51 

1 

72.0 

2 

32.4 

2 

52 

2 

73.1 

3 

33.6 

3 

52.9 

3 

74.3 

4 

34.7 

4 

53.8 

4 

75.5 

35 

35.8 

55 

54.7 

75 

76.7 

6 

36.9 

6 

55.7 

6 

78.1 

7 

38.0 

7 

56.7 

7 

79,6 

8 

39.0 

8 

57.8 

8 

81.0 

9 

40.0 

9 

58.8 

9 

82.5 





80 

84.0 
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TABLE 89. 

Abut Examination a: Distribution of PuPUiS in Grades 4, 5, 6, 7, 8, 9 

AND 13. 


Interval 

4 

n = 463 

5 

11 = 570 

^ 6 

11 = 672 

7 

11 = 685 

8 

11 = 630 

13 

n = 701 

0— 9 

4 






10-19 

9 






20-29 

16 

4 

1 

2 



30-39 

21 

5 





40 

36 

10 

2 


1 


50 

41 

21 

6 

1 



60 

46 

26 

3 

7 

2 


70 

57 

48 

15 

6 



80 

53 

45 

32 

11 

4 


90 

47 

64 

41 

20 

2 


100 

40 

53 

53 

27 

6 


110 

31 

61 

54 

36 

21 

1 

120 

22 

61 

64 

56 

30 

1 

130 

20 

55 

78 

55 

28 


140 

11 

43 

84 

63 

37 


150 

7 

28 

80 

65 

46 

3 

IGO 

1 

20 

53 

53 

52 

3 

170 

1 

11 

24 

79 

56 
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36 
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THE ARMY EXAMINATION A 

For special reasons, we have investigated the values of 
scores in the Army Examination a, although it is not in 
present use. AVe have distributions from Grades 6, 7, 8, 
and from college freshman, nearly seven hundred in each 
[Memoirs, p. 537], shown in Table 89. We shall also use to 

TABLE 90. 

Abmy Examination a ; Equiyalents for Each 10-Point Interval of the 
Original Scale in Equal Units. Results from Grades 6, 7 and 8. 


Original 

Interval 

Value in Equal Units 

Gt. 6 Gr. 7 Gr. 8 

Average 

60 to 69 

18.7 

18.95 


18.8 

70 to 79 

15.65 

6.9 


11.3 

80 to 89 

12.3 

9.15 

15.6 

12,4 

90 to 99 

11.6 

11.0 

4.0 

8.9 

100 to 109 

9.6 

9.7 

9.5 

9.6 

110 to 119 

10.1 

9.7 

17.1 

12.3 

120 to 129 

11.65 

11.4 

14.1 

12.4 

130 to 139 

12.65 

9.45 

9.2 

10.4 

140 to 149 

13.65 

9.6 

9.8 

11.0 

150 to 159 

11.3 

9.4 

10.05 

10.3 

160 to 169 

6,4 

7.65 

9.9 

8.0 

170 to 179 

8.65 

12.2 

9.9 

10.3 

180 to 189 

10.6 

10.05 

10.5 

10,4 

190 to 199 

5.2 

10.35 

9.8 

8.4 

200 to 209 

7.5 

13.2 

7.9 

9.5 

210 to 219 

9.4 

6.7 

9.6 

8.6 

220 to 229 

11.25 

9.6 

8.9 

9.9 

230 to 239 

0.00 

20.7 

10.05 

10,3 

240 to 249 

9.28 

7.3 

11.8 

9.5 

250 to 259 

13.48 

5.5 

8.45 

9.1 

260 to 269 



11.3 

11.3 

270 to 279 



12.9 

12.9 


some extent the distributions of 570 pupils in Grade 5 and 
463 pupils in Grade 4, which are also shown in Table 89. 

The values of each 10-point interval from 60 to 270 in 
equal units are computed for Grades 6, 7, and 8, by the 
methods previously used, and made strictly comparable by 
being divided by the difference between 90 and 230 of the 
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ariginal scale. They are then averaged. The true values 
of the intervals from 10 to 100 are computed for Grades 4 
and 5, assuming that the low ends of these distributions 
are distributed like the low end of Form A. They are made 
strictly comparable and averaged. The averages are then 
multiplied by a factor such that the difference 60 to 100 is 
represented by the same amount in the series of true values 


TABLE 91. 

Army Examination a: Equivai-ents for Certain 10-Point Intervam or 
THE Original Scale in Equal Units. Besults from Grades 4 and 5. 


Interval 

Values in 
Or. 4 

Equal Units 
Gr.5 

Av. 

At. X 1.28 

10 to 19 

16.4 


16.4 

21.0 

20 “ 29 

13,15 


13.15 

16.8 

BO 

10.1 

10.3 

10.2 

13.1 

40 

11.9 

10.5 

11.2 

14.3 

50 

10.1 

12.2 

11.65 

14.3 

60 

9.7 

9.5 

9.6 

12.3 

70 

10.8 

11.9 

11.35 

14.5 

80 

10.0 

8.6 

9.3 

11.9 

90 99 

9.7 

10.4 

10.05 

12.9 


obtained from Grades 6, 7, and 8, and in the series obtained 
from Grades 4 and 5. The 6, 7, 8 series is then extended by 
the values from 10 to 100 obtained from Grades 4 and 5, 
allowing equal weight to the two sets of values from 60 
to 99. 

The values of the intervals from 200 to 360 are com- 
puted from the facts for college freshmen (Grade 13). They 
are then multiplied by a factor such that the difference 200 
to 260 is represented by the same amount in the two series 
of true values (from Grades 6, 7, 8, and from Grade 13) 
The 6, 7, 8 series is then extended by the values for 260 to 
360 obtained from Grade 13. 

The essentials of these procedures and their results ap- 
pear in Tables 90, 91, and 92. 

A transmutation table in steps of 10 is then made, let- 
ting the two scales coincide at 170. This appears as Table 
93. 
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TABLE 92. 

Armt Examination A: Equivalents in Equal Units. Bebui/ts fbom: 
Grades 6 , 7 , and 8; 4 and 5; 13; and Composite from All. 


Original 

Interval 

Value 
By Grades 

6, 7 and 8 

in Equal Units 

By Grades 

4 and 5 

By 

Grade 13 

Composite 

1- 9 





10- 19 


21.0 


21.0 

20- 29 


16.8 


16.8 

30- 39 


13.1 


13.1 

40- 49 


14.3 


14.3 

50- 59 


14.3 


14.3 

60- 69 

18.8 

12.3 


15.6 

70- 79 

11.3 

14.5 


12.9 

80- 89 

12.4 

11.9 


12.2 

90- 99 

8.9 

12.9 


10.9 

100-109 

9.6 



9.6 

110-119 

12.3 



12.3 

120-129 

12.4 



12.4 

130-139 

10.4 



10.4 

140-149 

11.0 



11.0 

150-159 

10.3 



10.3 

160-169 

8.0 



8.0 

170-179 

10.3 



10.3 

180-189 

10.4 



10.4 

190-199 

8.4 



8.4 

200-209 

9.5 



9.5 

210-219 

8.6 



8.6 

220-229 

9.9 



9.9 

230-239 

10.3 



10.3 

240-249 

9.5 



9.5 

250-259 

9.1 



9.1 

260-269 

11.3 


8.7 

10.0 

270-279 

12.9 


9.2 

11.1 

280-289 



10.3 

10.3 

290-299 



9.7 

9.7 

300-309 



8.1 

8.1 

310-319 



10.3 

10.3 

320-329 



10.8 

10.8 

330-339 



7.7 

7.7 

340-349 



8.6 

8.6 

350-359 



9.7 

9.7 
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TABLE 93. 

Transmutation Table for Armt Exam. a. 


Original Seale in 

Scale Equal TTnita 


10 

~ 34.1 

20 

- 13.1 

30 

3.7 

40 

16.8 

50 

31.1 

60 

44.4 

70 

60.0 

80 

72.9 

90 

85.1 

100 

96.0 

110 

105.6 

120 

117.9 

130 

130.3 

340 

140.7 

150 

151.7 

160 

162.0 

170 

170.0 

180 

180.3 

190 

190.7 

200 

199.1 

210 

208.6 

220 

217.2 

230 

228.1 

240 

238.4 

250 

247.9 

260 

257.0 

270 

267.0 

280 

278.1 

290 

288.4 

300 

298.1 

310 

306.2 

320 

316.5 

330 

327.3 

340 

335.0 

350 

343.6 

360 

353.3 


TABLB 94. 

EquivaiiKnts POE A^my Exaukatiok a Scores prom 10 to 360 in a Scale wi t h Equal Units. 1 = 1/80 op the Dj 

Between 130 and 210 of the Original Scale. 
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From 130 to 360, the original scale value may be used 
with little error, but from 130 down the true values of the 
original scale units increase so that these 120 points of the 
original scale are equal to about 164 elsewhere. Table 94 
presents a detailed transmutation table made with some 
smoothing. 



CHAPTER VIII 


The Form of Distribution of Intellect in Man 
The orthodox doctrine is that the form of distribution 
of intellect in human beings of the same sex and age is 
Form A, shown in Fig. 15, representing a fact whose varia- 
tions up and down from its average condition are caused by 
a large number of uncorrelated factors each of which exer- 



cises about the same amount of influence on intellect as any 
other, and being a surface enclosed by a curve approximat- 

1 

ing the normal probability curve y= — where a is 

aV2TT 

the mean square variation. 

This doctrine "was urged by Francis Galton, on the basis 
partly of analogy with the facts in the case of certain bodily 
dimensions, and partly of his own shrewd observations of 
human abilities. Since his day it has gained very wide ac- 
ceptance. This is partly because the measurements of in- 
tellect and of other mental abilities in children of the same 
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age (their units being taken at their face value) have uni- 
formly shown continuity clustering around one mode, with 
diminishing frequencies in proportion to remoteness from 
that mode, and with no notable departure from symmetry 
toward any one special form of asymmetry. It is partly 
because some assumption had to be made in one investiga- 
tion after another for purposes of quantitative treatment, 
and this assumption was about as safe as any other one 
assumption, and much easier to operate with. Hence we 
gradually slid into the habit of using the doctrine. This 
fashion became so strong that in recent years psychologists 
have assumed symmetry, even though the units taken at 
their face value produced a markedly skewed distribution. 

GENERAL, CONSIDERATIONS 

Many of those who have made extensive use of this as- 
sumption have been aware of its highly hypothetical nature. 
The argument from analogy is weak because so many bodily 
variables are clearly skewed in distribution. Such are 
weight, longevity, girth of chest, strength of arm pull. The 
argument from mental measurements is weak, not only 
because of the general ambiguity of the units, but still more 
because the ‘‘error’’ has been a large proportion of the 
variation in many of the investigations. The “error” being 
symmetrical and “normal” tends to add a spurious sym- 
metry and normality to the variability. Moreover, some- 
times the selection is such that normality in the group mea- 
sured may well be an argument in favor of skewness for 
man in general. So, for example, vdth sixteen-year olds in 
high school, or twenty-five-year olds in universities. 

In general the form of distribution of any variable trait 
is due to the number of causes that influence variations in 
its amount, their magnitudes and their interrelations.^ 
Since we do not know what the causes of the variations in 

^ There is a certain regrettable vagueness, not to say ignorance, concerning 
the causation of variations, as when psychologists consider the amount of in* 
tellect to be a consequence of the presence or absence of a single Mendelian 
determiner, and yet to be distributed unimodally in Form A, Either of these 
beliefs really denies the other. 
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intellect in human beings of the same age are, we cannot as 
yet count them or measure their magnitudes or determine 
their correlations. We should then be very skeptical of a 
priori assumptions of Form A as the form of distribution 
of intellect in human beings of the same age. They are 
very much stronger in the case of children in the same 
school grade. 

In the case of children in the same school grade the 
causes are our own acts; and we do know that school au- 
thorities have a rough standard of the educational ability 
which belongs in a certain grade, say Grade 7, that intellect 
correlates closely with educational ability, that departures 
from this standard (that is, mistakes in grading) are rare 
in some proportion to their magnitude, that they are due 
to many causes (the different teachers’ judgments with all 
the experiences upon which they are based, and the ideals 
and prejudices which they exemplify, and the other causes 
of error to which they are subject), and that many of these 
causes are only loosely inter-correlated. These are all fea- 
tures of a status productive of symmetry and normality. 

In the case of children of the same age (or age and sex 
and race) the causes are acts of nature, many of them hap- 
pening millenniums ago ; and we do not even know whether 
the hereditary factors of variability in intellect are six big 
ones or sixty small ones. We do not know whether the 
words heard and acts seen in the first three years of life are 
of almost zero consequence, as used to be thought when 
favored children w^ere turned over to healthy peasants dur- 
ing this period ; or are of enormous consequence, as is as- 
serted by Freud and (but for different reasons) by Wat- 
son.“ So we may best consider the facts of the distribution 
of intellect in man with little or no pre-disposition. 

2 There is one special set of major causes of variation about whose action 
we do know something. Certain diseases and certain accidents, either before 
or during or after birth, act to prevent or reduce the development of intellect. 
In some cxises one of these causes may act to prevent intellect from reaching 
more than a certain very lowly status regardless of what might have happened 
had its action been withheld. The result may be that whatever the distribu- 
tion apart from these causes, there is combined with it a very small distribu- 
tion with a mode at a very low degree of intellect, as shown in Fig. 16. 
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National Intelligence Examination: Distbibdtion of Scores for White Pupils, Age 11. 
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THE EVIDENCE 

The results stated in Chapter VII permit us to free the 
evidence of the past from the ambiguity and misleading of 
units whose real value was unknown. If we had the time 
and facilities, we could free them also from the constant 
tendency toward symmetry and normality due to the error 
of measurement, but that work must be delayed. What we 
can do now is to show the form of distribution of children 
of the same year-age in respect of intellect in so far as it is 
measured by the Haggerty or by the Otis or by the National 
A, and in so far as the children examined are a random 
sampling of the children at that age. We do not separate 



the sexes, because the sex differences are small and the 
separation would leave us with too small populations. We 
do not separate races, because that cannot be done in the 
records available. Negro schools are very rarely, if ever, 
included in the records ; but negro children and children of 
mixed parentage doubtless are sometimes reported without 
distinction, and so included in the distributions. 

The ages used are 11, 12, 13, and 14, at which years cer- 
tain very dull children have been excluded from school at 
home or in institutions. Some 14-year olds have left school. 
The measurements were taken in schools, so that there is 
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too small a representation of truants and of sickly children. 
City children are more fully represented than country chil- 
dren. The schools are predominantly public schools; so 
that Catholic children are insufficiently represented. The 
age is doubtless sometimes in error, and is a year wide. 
The latter fact should spread and flatten all the distribu- 
tions a little. 



National A, transmuted into a scale with equal units* 



It seems unwise to tamper with the records in an effort 
to allow for these various factors and make the distribu- 
tions more exactly representative of ‘‘all white children of 
the United States of age x.^’ The process of allowance 
would probably make improvements, but they would be 
small and uncertain and very tedious to make and to under- 
stand. So we shall take the facts just as Haggerty, Otis, 
and the National Committee give them; and do nothing to 
them save transmute each scale interval into units which 
are truly equal, construct the resulting distributions, and 
measure certain of their properties. In our inferences from 
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the results we shall, of course, try to hear all the condition- 
ing factors in mind. 

Consider first Tables 95 to 98, which give the facts in 
the case of the National A. Tables 95, 96, and 97 give the 
original data. Table 98 gives the data for constructing the 
surfaces of frequency in the shape of the true values for 
each interval, which are taken as lengths along the abscissa 
line, and the quotients of the permille numbers each divided 
by its corresponding abscissa length. These quotients give 




Fig. 20. An approximate composite of Figs, 17, 18, and 19. 


the relative magnitudes of the ordinates or heights of the 
rectangles erected over the corresponding abscissa lengths. 
Figs. 17, 18, and 19 show the resulting surfaces of fre- 
quency with equal units. Fig. 20 shows a rough composite 
picture of the form of distribution of National A ability in 
children of the same year-age. It contains the three sepa- 
rate distributions centered on their medians. 

Tables 99 and 100 show the original data in the case of 
the Otis Advanced Examination. Table 101 shows the ab- 



National Intelligence Examination A: Distribution of Scores for White Pupils, Age 13. 
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TABLE 98. 

Kationaii a. Data for Stirfacjc of Fbrquengy in Equal Units. 


Original 

Interval 


Abscissa 
Liength in 
Equal 
Units 


Ordinate Heights to Make the Areas Equal to 
the Corresponding Permille Entries of 
Tables — 95, 96 and 97 
Age 11 Age 12 Age 13 


10-19 

10.00 (Est.) 

5 

2 

2.5 

20-29 

9.61 

12.5 

6.2 

1.6 

30 



10.8 

7.5 

40 

8.63 

31.3 

25.5 

17.4 

50 

7.92 

59.3 

42.9 

35.4 

60 

8.12 

99.8 



70 

8.55 


93.6 

59.6 

80 

8.32 

137.0 

93.8 

81.7 

90 

9.85 

KB 

112.0 

93.4 


9.44 



118.0 

110 




128.0 

120 

8.95 

107.0 

115.0 

145.0 

130 

9.92 

72.6 

85.7 

106.8 

140 

11.14 

32.3 


86.3 

150 

13.37 

14.2 

44.1 

54.5 

160 

14-59 

4.8 

15.8 

28.8 

170 



5.3 

4.7 

180 

16.00 (Est.) 



0.7 


scissa lengths in equal units and the ordinate heights ob- 
tained by dividing each original permille number by the 
corresponding abscissa length in equal units. Figs. 21, 22, 
23, and 24 show the surfaces drawn according to Table VII. 



Fig. 21. The form of distribution of the scores of ll-year-old children in Otis 
Advanced, transmuted into a scale with equal units. 


20 





280 


THE MEASITBEMENT OF XNTEIXIQEKCE 









THE FOBM OF DISTRIBUTION OF INTBULECT IN MAN 281 


Pig. 25 is a composite repeating Figs. 21 to 24, with the 
four medians coinciding. 

Table 102 shows the original data for the Haggerty 
Delta 2; Table 103 shows the lengths and heights when 



Hagg^erty Delta Two, transmuted into a scale with equal units* 





TABLE 99. 

Otis Advanced Examination: Distbibution of Scores: Ages 11 and 12. 
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equal units are used. Figs. 26, 27, 28, and 29 are the result- 
ing surfaces of frequency; Fig. 30 is their composite. 

Fig. 31 is a composite of the three composites. Figs. 20, 
25 and 30. 





Fio. 31. A composite of three composites. 






TABLE 100. 

Otis Advanced EzAiiiNAnON: DisTsiBimoN or Scokes: Ages 1? akd 14. 
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The ability which is measured by the score of any one 
of the commonly used intelligence examinations is thus 
shown to be distributed in children of the same age (from 
11 to 14 in rather close approximation to Form A. There 
are no demonstrable departures from unimodality or from 

TABLE 101. 

Otis Advanced Examination: Data by Which the Surfaces of 
Frequency Are Constructed. 


Intervals 
by Original 
Scores 


Values of 
Intervals in 
Equal Units 


Heights of the Surface of Frequency with 
Equal Units to Make the Areas Equal 
to the Corresponding Permille 
Entries 




11 

12 

13 

14 

0- 9 

20.0 (Est.) 

1.0 

0.5 



10-19 

18.3 

1.1 

2.2 


.6 

20-29 

14.8 

10.8 

4.7 

6.6 

2.5 

30 

12.2 

26.2 

27.9 

9.8 

13.4 

40 

10.9 

64.2 

45.0 

28.4 

16,0 

50 

10.7 

116.9 

67.3 

55.2 

41.7 

60 

10.4 

110.5 

96.2 

74.1 

56.5 

70 

10.0 

150.0 

112.0 

96.0 

52.2 

80 

9.7 

142.3 

170.0 

105.0 

77.2 

90 

9.7 

112.4 

137,0 

112.0 

98.9 

100 

9.5 

98.9 

103.0 

126.0 

110.9 

110 

9.5 

75.8 

85.3 

99.0 

125.0 

120 

9.6 

37.5 

44.8 

98.0 

123.9 

130 

9.6 

19.8 

40.7 

95.9 

99.7 

140 

9.6 

8.3 

28.2 

49.0 

81.5 

150 

10.5 

9.5 

21.0 

29.6 

58.0 

160 

11.0 

1.8 

7.3 

17.3 

29.7 

170 

11.2 


3.6 

4.5 

14.6 

180 

11.8 


.9 

1.7 

4.7 

190 

16,0 



.6 

2.1 


symmetry; the decrease in frequency as we pass from the 
mode is slow, then more rapid, and then slow again. 

THE FORM OF DISTRIBUTION AT AGES UP TO FIFTEEN 

It is reasonable to infer that the form of distribution 
which is found for these examination scores, when trans- 
formed into a scale with equal units, will be found with very 
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little change for any valid measures of the altitude of In- 
tellect CAVD, or of Intellect GOPI (letting G refer to geo- 
metrical tasks, O to opposites, P to picture completions, I 
to information), or of any representative sampling of intel- 
lectual tasks. It is reasonable to carry the inference on to 
any valid measures of the histological and physiological 
basis of altitude of intellect. It is probably safe also to ex- 

TABLE 102. 

Hagoerty Delta 2; Distribtjtion of Scores. Data from Mai>sen [’22], 


Age 11 Age 12 Age 13 Age 14 

Grades 3 to 8 Grades 3 to 10 Grades 3 to 10 Grades 3 to 11 



n 

Per- 

mille 

n 

Per- 

mille 

n 

Per- 

mille 

n 

Per- 

mille 

0- 9 

5 

6 

4 

5 



1 

2 

10- 19 

6 

7 

8 

10 

4 

5 

3 

5 

20- 29 

8 

10 

6 

7 

6 

8 

2 

3.5 

30- 39 

39 

48 

22 

28 

11 

14 

9 

6 

40- 49 

78 

96 

35 

44 

16 

21 

11 

19 

50- 59 

80 

98 

62 

79 

26 

34 

13 

23 

60- 69 

99 

122 

74 

94 

48 

62 

32 

57 

70- 79 

106 

130 

92 

117 

65 

85 

37 

66 

80- 89 

106 

130 

93 

118 

88 

114 

66 

117 

90- 99 

100 

123 

107 

136 

99 

129 

76 

135 

100-109 

89 

109 

95 

121 

115 

149 

83 

147 

110-119 

44 

54 

69 

88 

112 

145 

77 

136 

120-129 

36 

44 

65 

83 

76 

99 

76 

135 

130-139 

17 

21 

33 

42 

68 

88 

47 

83 

140-149 

1 

1 

18 

23 

26 

34 

28 

50 

150-159 

1 

1 

3 

4 

7 

9 

4 

7 

160-169 





2 

3 



170-179 



1 

1 

1 

1 



Total 

815 


787® 


770 


565 



tend the inference back to ages eleven to one, since there is 
no evidence that mortality from one to twelve is selective 
in respect of altitude of intellect to any considerable ex- 
tent, or that the environment acts during those years to 
reduce and counteract tendencies to multimodality, skew- 
ness, and other departures from Form A. With a little less 
assurance, we may extend it back to the germ cells and 
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assert that, to a close approximation, the original capacities 
of white children in the United States to manifest given 
altitudes of intellect are distributed in a surface that is ap- 
proximately unimodal, symmetrical, and of Form A. 

THE FORM OF DISTRIBUTION IN ADULTS 

Extending the inference to later ages is a very different 
matter. If the distribution is “normal’’ at 14, it may still 
become skewed at 24. This would happen if the gains made 
by those of different degrees of intellect at 14 differed in 
certain ways and by certain amounts. For example, sup- 
pose that the altitude of intellect of fourteen-year-olds is 
distributed as shown in column I of Table 104, and that 
from fourteen to twenty-four those individuals of abilities 
1, 2, 3, 4, and 5 gain 0, while those of abilities 6 to 19 gain as 
shown below: 


Ability 

6 

0 

Gain 

to 

.15 

7 

•15 

to 

.35 

8 

•35 

to 

.55 

9 

.55 

to 

1.3 

10 

1.3 

to 

2.0 

11 

2 

to 

6 

12 

5 

to 

10 

13 

10 

to 

16 

34 

16 

to 

40 

15 

40 

to 

80 

16 

80 

to 

150 

27 

150 

to 

300 

IS 

300 

to 

600 

19 

600 

to 

1000 


The distribution at age twenty-four would then have its 
low extreme at 1 as before, its mode and median at about 
11, and an enormous skew running up to about 1,000. To 
take a much less extreme state of affairs which might be 
real, suppose the condition at fourteen to be as in columns 
II and III of Table 104, and the gains to be as shown in 
column IV. Then the condition at twenty-four will be as 
shown in column V with a clear skew. 
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We also have evidence that a positive relation of gain 
to ability exists in the case of the ages above fonrteen, 
though we do not know its exact nature or amount. Imbe- 
ciles notoriously gain very little. Thorndike has shown 
[*23] that the sort of pupil who attends high school gains 
up to eighteen at least, in the ability measured by stock 
intelligence tests, and that the white pupils gain much more 
than the colored pupils. 

TABLE 103. 

Haogertt Delta 2. Data roa Surface op Frequency in Equal Units. 


Original 

Interval 

Abseissa 
Length in 
Equal Units 

Ordinate Heights to Make the Ar< 
Equal to the Corresponding Per 
mille Entries of Tables 

Age 11 Age 12 Age 13 Age 1 

0^ 9 

12,00 (Est.) 

5 

4 


2 

10-19 

12.00 (Est.) 

6 

8 

4 

4 

20-29 

10.70 

9 

7 

7 

3 

30 

9.50 

51 

29 

15 

6 

40 

9.33 

103 

47 

23 

20 

50 

10.85 

90 

73 

31 

21 

60 

10.55 

lie 

89 

59 

54 

70 

8.62 

151 

136 

99 

77 

80 

9.48 

137 

124 

120 

123 

90 

9.15 

134 

149 

141 

148 

100 

10.54 

103 

115 

141 

139 

110 

10.01 

54 

88 

145 

136 

120 

12.06 

36 

69 

82 

112 

130 

13.01 

16 

32 

68 

64 

140 

19.86 

0.5 

12 

17 

25 

150 

20.88 

0.5 

2 

4 

3 


The differential gain could be caused by several differ- 
ent factors. Inner mental growth is less in amount in the 
dull at ail ages; it may, and probably does, slow up and 
approach zero earlier in the dull. Insofar as ability with 
intellectual tasks is due to environment and training, the 
expectation will be that each added acquisition will be a 
stimulus to others and an aid in acquiring them. So learn- 
ing to read commonly leads to the acquisition of a wider 
vocabulary and a better score in opposite tests and comple- 
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tion tests than would have heen attained by oral intercourse 
alone. The more intellectual the individual is, also, the 
more will he give his free time to intellectual pursuits. 
Finally, vocational selection is such that the more intcUee- 

' TABLE 104. 

The Effect of OoRXEiiATiON Between Status and Gain When Gain In- 
creases IN A Geometric Ratio. 


I 

Status 

II in 

Frequency at 14 
Grouped Grouped 

by 1 'a by 3 'a 

IV V 

Gain Frequency at 24 

14 to 24 

0 





1 

.02 

.2 

.1 + 

0.2 

2 

.19 


.125 


3 

1.14 


.156 


4 

4.85 

21.5 

.195 

21.5 

5 

15.50 


.244 


6 

38.76 


.305 


7 

77,52 

242 

.381 

242 

8 

125.97 


.477 


9 

167.96 


.596 


10 

184.76 

521 

.745 

510 

11 

167.96 


.931 


12 

125.97 


1.16 


13 

77.52 

242 

1.45 

214 

14 

38.76 


1.82 


15 

15.50 


2.27 


16 

4.85 

21.5 

2.84 

53 

17 

1.14 


3.55 


18 

.19 


• 4.44 


19 

.02 

-2 

5.55 

6 

20 





21 





22 




0.2 

23 





24 





25 




0.02 


tual individuals continue in school and engage in clerical 
and professional work that involves intellectual activities, 
while the dull leave school for labor which requires little 
thought, and sometimes does not even permit it. 
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There may of course be a marked increase of gain for 
those of higher abilities without producing skewness. If 
g = as -f- b, no skewness will be produced, no matter how 
steep the relation line may be. The variability will in- 
crease, but the form will still be Form A, as shown in Table 


TABLE 105. 

The Effect op Cokrelation between Status and Gain when g = as + b. 



1 
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105, where the abilities 1, 2, 3, 4, 5, 6, etc., have gains of 2, 
4, 6, 8, 10, 12, etc. 

The causes which influence differences in gains in intel- 
lect up to about fourteen do seem to produce them in a 
rough proportion to the differences in ability, so that the 
form does remain that of Form A. 

The only data that we have found for measuring the 
form of distribution of anything approximating a random 
sampling for any age above fifteen are the well-known 
Army records with Alpha, Beta, and Examination a. We 
have no satisfactory means of determining the value of 
Beta scores in a scale of equal units. So we limit our in- 
quiry to Alpha and Examination a. 



transmuted into a scale with equal units. 

Using the equal-unit values for Alpha and a derived in 
Chapter VII, and proceeding as in the case of the National, 
Otis, and Haggerty scores for children, we obtain the re- 
sults shown in Fig. 32 for 51,620 native-born whites of the 
draft LBata from the National Academy of Sciences Mem- 
oirs, '21, p. 764]. 

The equal-unit values of the interval from 0 to 20 in 
Army Alpha and from 0 to 30 in Examination a are esti- 
mates from exceedingly scant data. 
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The lower end of Fig. 32 would be extended if the illit- 
erates who were exempt from Alpha had been included. It 
would have been extended still further if the men rejected 
for dullness by the examining boards had been included. 
The upper end would be extended if the officers had been 
included. 

Using the equal-unit values for Examination a in the 
same manner, we obtain the results shown in Fig. 33 for 



tion a, transmuted into a scale with equal units. 

63,647 enlisted men in four camps. [Data from the Mem- 
oirs, ’21, p. 492.] The same considerations concerning the 
inclusion of men rejected for dullness and of officers apply 
as applied in the case of Alpha. In these four camps, 13.9% 
had been excluded from examination as illiterate. 

It is difficult to reach any secure conclusion from the 
facts of Fig. 32 and Pig. 33, except that there is no evidence 
of negative skewness. From Alpha alone in the general 
draft it would appear that even after generous allowance 
for the dullness of the illiterates and others who were ex- 
cluded from examination, the distribution was skewed posi- 
tively, i.e., toward the high end. With Examination a in 
the four camps, however, the skewness of the original scores 
disappears when the values in equal units are used. 
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We cannot even estimate with surety what the distribu- 
tion of 51,620 of the native-born whites would have been if 
they had been measured with Examination a, or what the 
distribution of the 63,647 enlisted men in the four camps 
would have been if they had been measured with Alpha. 
That is, w^e cannot decide how far the difference between 
Fig. 32 and Fig. 33 is caused by the tests used and how far 
it is caused by the groups tested. 

On the whole,® we may provisionally regard the sort of 
intellect measured by Alpha and a as distributed in the 
adult native-born white population of the United States 
with some positive skevmess. We may provisionally as- 
sign, as the cause of the change from the symmetry and 
normality found in children, a differential gain from the 
age of fourteen to twenty and beyond, whereby some indi- 
viduals increase these abilities very greatly, whereas others 
increase them little or not at all. This should be only pro- 
visional. The whole matter of adult intellect should some- 
time be studied with the care which it deserves. 

For the ages from 14 up to 17 or 18, we may assume 
symmetry and normality w’ithout much probability of more 
than a small error. Or, we may have a slightly greater 
prospect of correctness if we allow for a very little positive 
skewness, increasing year by year. 

s Certain facts of the distribution of men in occupations, of the distribution 
of vr&ges, of the distribution of schooling, and the like rather favor the sup- 
position that adult intellect is distributed with positive skewness. 



CHAPTER TX 


A Scale for Measuring Altitude of Intellect 

It was not a part of our original plan to make an actual 
scale for measuring intellectual difficulty, but only to find 
methods whereby this could be done. We have proved that 
the form of distribution of altitude of intellect in grade 
populations from Grade C to the first year of college can be 
known, so that the degree of intellectual difficulty of a com- 
posite task which is truly intellectual can be measured by 
the percentage of successes in such a grade population. 
We have also shown that the form of distribution of intel- 
lect of an age population 10 to 14 is approximately of Form 
A, that of the normal probability surface, so that the same 
procedure can be followed in one of these age groups. It 
is highly probable that it can be followed in lower age 
groups. 

Although we did not plan for scaling the difficulty of 
actual tasks and are not able to do it jirecisely witli the time 
and facilities at our disposal, it seems best to make a begin- 
ning, if only to illustrate the workings of the principles and 
techniques involved in an actual case. 

The work on this scale may best be considered in two 
parts, that which evaluates the differences in difficulty of 
Composites I to Q, and that which evaluates the differences 
in difficulty of Composites A to I. The latter was done 
primarily to put the values for I, J, K • ■ ■ Q in relation to 
the absolute zero, for which purpose chance errors in the 
determinations of B-A, G-B, D-C ‘ • * I-Il are of minor 
importance, since they tend to equalize one another. These 
lower intervals are less precisely determined than those 
from I to Q ; and we report them and their derivations sepa- 
rately in the latter part of the chapter. 
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THE DIFFICULTY OF COMPOSITES I, J, K, L, M, H, O, P, AND Q 

. We present first the measurement of differences in dif- 
ficulty between tasks I and J, between J and K, between K 
and L, and so on with M, N, O, P, and Q. The facts at our 
disposal for the measurement of differences in difficulty 
amongst these composite tasks are the percentages correct 
in various groups as shown in Table 106. Group 5i refers 
to the 147 pupils measured at the end of Grade 5 and at the 
beginning of Grade 6 with composites I, Jj and K. Group 
91 refers to the 246 pupils of Grade 9 who are measured 
with composites I, J, K, Li, and M. Group 9II refers to the 
192 pupils of Grade 9 who are measured with composites 

TABLE 106. 

Percents of Various Groups Succeedino with 20 or Moke Single Tasks 

OF CAVD 40 -Composites I to Q. 


Composite 


Percents Succeeding 

51^ 91 9II 13 17 


I 

91.2 

99.6 

J 

29.1 

89.4 

K 

li.O 

61.4 

L 


32.9 

M 


5.3 

N 



O 



P 




Q 


47.0 

16.3 

7.2 


1.1 

81.5 

95.4 


48.1 

77.1 


27.3 

56.7 


3.7 

22.9 


K, L, M, and N. Group 13 refers to the 189 candidates for 
entrance to college who were measured w'ith composites N, 
O, P, and Q. Group 17 refers to the 240 college graduates 
who were measured with JST, O, P, and Q. 

If we know the form of distribution of a group and the 
percent of the group succeeding with a task, it requires 
only straight-forward mensuration to find the point on the 
base line corresponding to that percent, and the distance of 
that point plus or minus from the median (or mode, or 
other point of reference defined by the distribution of the 

21 
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group) in terms of the mean square variation (or other de- 
fined measure of variability) of the group in whatever abil- 
ity is measured by that task. 

The form of distribution is taken as normal for each 
of the grade groups, 5^, 91, 9II, and 13, in consequence of 
the facts outlined in Chapter II and presented in detail in 
Appendix III. The form of distribution of Group 17, which 
was composed of first-year law-school students, all college 
graduates, was determined by a special investigation which 
is reported in Appendix VI. The same has been done for 

TABLE 107. 

The Difficulty of Composites I to Q in Various Groups Expressed in 
Each Case as a Deviation prom the Difficulty for the Median of 
THAT Group, in Terms of the a op that Group in the Ability 
Measured by Success with the Composite in Question. - is 

Easier, + is Harder. 


Composite 



Difficulty 




5^ 

91 

9II 

13 

17 

I 

-1.35 

-2.65 




j 

+ .55 

-1.25 




K 

+ 1.20 

- .29 

+ .08 



L 


+ .44 

+ .98 



M 


+ 1.62 

+ 1.46 



N 



+ 2.29 

- .897 

- 1.862 

O 




+ .048 

- .714 

P 




+ .598 

- .153 

Q 




+ 1.787 

+ .738 


the form of distribution of certain groups used, later in this 
chapter, namely, for the 180 adult imbeciles of mental age 
from 2i to 5 years, for the 100 adult feeble-minded of men- 
tal age near 6^, for the group of 50 feeble-minded at or near 
mental age 8, for the group of 101 dull pupils 13 years old 
or over, in special classes in New York City, for the popu- 
lation of Grade 4 (second half year) and for the population 
of Grade 5. The evidence and argument in all these cases 
appear in Appendix VI. 

Table 107 gives the difficulty of various 40-composites 
in various groups, expressed in each case as a distance 
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from the difficulty which 50% of that group can succeed 
with and in terms of otj (the mean square deviation of 
whatever ability is measured by that composite in that 
group). The next procedure in constructing a scale of dif- 
ficulty is to make all these different measurements of dif- 
ficulty commensurate and put them all into relation to the 
same point of reference. This is a complicated procedure 
involving the following steps : 

ICach measurement in ot^ for a given group is to be 
turned into a measurement in <ji for that group, Oi being the 
mean square variation of the group in altitude of intellect 
perfectly measured in truly equal units. 

Each measurement in the oi of a certain group must 
be made commensurable with measurements in the o, of 
any other group, by finding the comparative magnitudes 
of Oi of the 240 graduates, Oi of the 189 college entrants, 
a, of the 246 pupils in Grade 91, oi of the 192 pupils in 
Grade 911, and so on. All of the different Oi values may 
then be multiplied or divided by numbers so that all will be 
expressed in the same units. We shall use the mean square 
deviation of pupils in Grade 9 as our unit for this purpose. 

The measurements, now in units of 0 , 9 , must be ex- 
pressed, not as distances plus or minus from the CAVD 
difficulty for the median now of one group now of another, 
but all from some common point of reference such as the 
median for Grade 9. 

ESTIMATIZfG Oi FROM <Jt^ 

We turn the measurements in oti into terms of Oi by 
using Oi = ot, Vrt,t, or by using o, = ot^ rt,i. 

The self-correlation rtit 2 is, of course, for a 40-composite 
with another 40-composite of equal difficulty, not for an in- 
finitely extensive set of tasks of a certain difficulty with 
another equally extensive set. Also rtit 2 is the correlation 
for the specific group of restricted range which is being 
used, not the correlation for a group of wide range, such 
as all persons of age 20 . 
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For precise determinations of or of rt,i we need 

measnrenients with more extensive groups and alternate 
forms of our 40-composite tasks. With the material at our 
disposal we can hope only for approximate results. 

We measure or infer rtit 2 separately for each composite 
with each group. We may, however, wisely modify the esti- 
mate for each composite with each group in view of the 
facts concerning rtit 2 for the same composite in other 
groups, or for other neighboring composites in the same 
group. 

Consider, for example, the 40-eomposites K, L, and 
M. The correlations of each of these with a 40-composite 
of diiferent content but similar difficulty estimated by 

_i_ ° — are as shown below according to the group and 

"i ^20, 20 

kind of coefficient computed. 

K L M 

Group 246 (Sheppard) ,74+) , 

Group 246 (Pearson) .65 + ) ‘ 


.70 


Group 192 (Sheppard) .80 '<' ) 77 
Group 192 (Pearson) .73 + f ' 


.86 j 

1 .86 

1 

.69 

. 86 ] 

.68 

. 86 ] 
.89 1 

[.87% 

.73 

.78 


. 68 % 


■'|}.7514 


The correlations of each of them with a 40-composite of 
different content but similar difficulty may also be esti- 
mated by adding .03^ to their average correlations with 
their nearest neighbor composites (or, with some justifica- 
tion, by adding .02 or .01 or even 0). Using .03, we have 
the results shown below. 


Group 246 (Sheppard) 
Group 246 (Pearson) 

.70] 
.69 J 

K 

[ .69% 

.68 ] 
.69 J 

L 

[ . 68 % 

M 

Group 192 (Sheppard) 
Group 192 (Pearson) 

.50 1 
.64 ] 

1 .57 

1 

.65 1 
.70 ] 

- .67% 



Combining the two sorts of estimates, we have 



K 

L 

M 

Group 246 

.70 

.77 

.68 

Group 192 

.67 

.77i 

.75 


^ See Appendix lY for the deriration and justification of this allowance 
for remoteness. 
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Moreover, we may consider that chance played some 
part in making the self-correlation of L higher than the 
other two ; and so lower its rtita and raise theirs somewhat 
to balance. Similarly we may consider that chance played 
some part in making these Ttit,’s higher in the 192 group 
than in the 246 group, and allow somewhat for that. Thus 
we may replace the last set of figures by 



K 

L 

M 

Group 246 

.72 

.76 

.70 

Group 192 

.67 

.7^ 

.75 


in which slight smoothing by these allowances is made. 

In Table 108 is collected all the information concerning 
the rt^ta’s for each 40-composite in each group. I and II 
refer to the two methods of determining rtit 2 - In I we use 
the correlation between the two halves of a 40-composite 
obtained bj’^ taking 5C + 5A -j- 5V + 5D at random, the sec- 
ond half being composed of the remaining 5C + 5A + 5V + 

^^20 20 

5D ; and estimate r 40 , 4 o by / . That is, rtit 2 = 

-L ~r 5*20, 20 

2T20, 

. In II we use the obtained correlation between 

" 7 “ 20 

the 40-composite in question and its nearest neighbor com- 
posites,^ adding .03. 

The correlations for composites N, O, P, and Q in group 
17 under “By other data** were obtained as follows: A 
composite almost identical with N was correlated with an- 
other of very closely equal difficulty, giving r = .72. A com- 
posite almost identical with O was correlated with a com- 
posite of very closely equal difficulty, giving r = .75. The 
composite almost identical with N -was also correlated with 
a composite almost identical w^ith O, giving r = .73. The 
composite almost identical wdth O was correlated with Q, 
giving r = .73. From these correlations, allowing + .03 for 

* The results by method I are in general higher. The differeneea (Method 
I -Method II) are; .09 .12 0 .17% .00% .20 .20 .00% .0i% -.07 .05 .05 
- .06, averaging .063. 
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one step of remoteness, we have the self-correlation of N 
as .72 or .76, averaging .74; that of O as .75 or .76, aver- 
aging .75^ ; and that of Q as .76. 

91 and 9II differ almost nil in the general magnitude of 
rtit 2 for the three 40-composites used with both of these 
groups, the average difference (I-II) being — .013 with a 


TABLE 109, 

Values or Debived feom Table 108, and the Values or Used 

TO Obtain Table 110 erom Table 107. 



5 ^ 

91 

*■*1*2 

9II 

13 

17 

5% 

91 

9II 

13 

17 

I 

.78 

.66^ 




.883 

.815 




J 

.78 

•72% 




.883 

.851 




E 


.68% 

.68% 




.828 

.828 



L 


.77% 

.77% 




.880 

.880 



M 


.70 

.73 




.837 

.854 



N 



.72 

.72% 

.77 



.848% 

.851% 

.877% 

O 




.77 

.82% 




.877% 

.908 

P 




.77 

.81 




.877% 

.900 

Q 




.69 

.74 




.831 

.860 


mean square error of ± .033, three times as great as the 
difference. So we shall probably be nearer the truth by 
using .68^ and .68J in place of the .70 and .67, and .70 and 
.73 in place of the .68 and .75. 

In general rtito is .041 higher in 17 than in 13, and the 
use of this fact to smooth out the irregularities in the values 
for N, O, P, and Q will probably be an improvement. Thus, 
columns 3 and 4 below are probably truer than column 1 
and 2. The totals for each group and for each composite 
are unaltered by the amendments. 


From the table Amended 



1 

2 

3 

4 

N 

691 

80 

72i 

77 

O 

78| 

81 

77 

82i 

P 

78i 

79i 

77 

81 

Q 

69 

74 

69 

74 
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We make the amendments noted in the last two para- 
graphs and so use the rtit,*s listed in Table 109 in estimat- 
ing the difficulty in terms of <ji for each composite in each 
group. The results are shown in Table 110. 

TABLE 110. 

The iNTEIiLECTTTAL DIFFICULTY OF COMPOSITES I TO Q IN GSOUPS 5%, 91, 9II, 
13 AND 17. Expressed in Teems of oi o, Ot u, cii niy Oi n ob oi 
AS Derived by the Use of Table 109. 


Com- 

posite 

In Oi ti 

lu 0*1 »1 

Difficulty 

111 Cl Bit 

I®^ Oi » 

In Oi n 

I 

-1.53 

-3.25 




J 

+ .62 

-1.47 




K • 

+ 1.36 

- .35 

+ .10 



L 


+ .50 

+ 1.11 



M 


+ 1.94 

+ 1.71 



N 



+ 2.70 

- 1.054 

- 2.120 

O 




+ .055 

- .786 

P 




+ .681 

- .170 

Q 




+ 2.150 

+ .858 

Estimating 

the Oi’s 

by ai = at, rt,i, 

we obtain 

rt,i by 


*■ 1*1 . 

rt,i — _^=r- in which rt.i, is the obtained correlation be- 

tween the 40-composite in question and the summation score 
in a long CAVD series, and ri,i, is the self-correlation of 
this summation score. In certain cases we have to estimate 
Fiji,, but the error of the estimate is small,® and its effect is 
reduced since only the square root of rtji, is used. The 
values of rt^t^ and ri,i, used are those used for another pur- 
pose in Appendix V- The results of the computations are 
shown in Table 111. 

Using the estimates of rt,i of Table 111, we obtain the 
estimates of the difficulty of each composite for each group 
in terms of the o, of that group which are presented in 

3 For a grade population the empirical values of vary from .91 to .95. 
In group 5% and group 17 where we estimatei the summation score is from 
a very long series, so we use .95. 
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Table 112. These differ on the average from those of Table 
110, as shown below, the median difference regardless of 
signs being .03 and the average difference .05. 

Gi (by Vrtit 2 ) — Oi (by rtn) 


— .15to — .06 4 

— .05 to + .04 12 

+ .05 to -j- .14 3 

+ .15 to + .24 1 


TABLE 111. 

VAiiiTEs OP Estimated fhom Cokrelations between Number op Singxjb 
Tasks Correct in a 40-Composite and Number Correct in a Long 

CAYD Series. 


Composite 

0% 

91 

9II 

13 

17 

I 

.933 

.759 




j 

.882 

.907 




K 


.854 

.819 



L 


.944 

.896 



M 


.849 

.922 



N 



.819 

.824 

.872 

O 




.917 

.944 

P 




.948 

.913 

Q 




.790 

.882 


Except in the case of composite I in group 91, ‘it does 
not matter much whether we use the estimates of Table 110 
or those of Table 112 or averages of the two. We have 
averaged each pair of determinations with the results 
show'll in Table 113 Avhich are used as the ai values in all 
that follow^s. 

EXPRESSING THE Oi OF EACH GROUP IN TERMS OF A COMMON 

UNIT 

We make the oi ’s of tw^o groups, A and B, commensurate 
by finding the difference in difficulty between twro tasks in 
terms of oia in terms of 0 ,b, provided the two groups 
overlap sufficiently. Thus, we find, in the case of the group 
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(17) of 240 college graduates and the group (13) of high 
school graduates, that: 

Composite O — Composite N = 1.12ai is and 1.36ai it* 
Composite P — Composite O — .61oi is and .GOoi it* 
Composite Q — Composite P = 1.55oi is and 1.02o, 17. 

Oi is==l*21oi IT or .99ai IT or .6601 it, according to the 
successive pair of composites used. If we take the most 
remote composites which include all the data, Q and N, we 
have 3.28ai i3 = 2.98ai it, whereby oi i3 = .91ai 17. 


TABLE 112. 

The iNTEtiiaECTUAL Difficulty of Composites I to Q in Terms of ci ii, 
Oi at , ETC.; As Derived by the Use of Table 111. 


Composite 

In 0, 4 

Oi 01 

Difficulty 

In Ot 911 

In 01 u 

In 0, 1, 

I 

^1.45 

-3.49 




J 

+ .59 

-1.38 




K 

■f 1.29 

- .34 

+ .10 



L 


+ .47 

+ 1.0914 



M 


+ 1.91 

+ 1.58 



N 



+ 2.80 

-1.09 

- 2.1314 

0 




<- .05 

- .76 

P 




+ .63 

- .17 

Q 




+ 2.26 

+ .84 


In the same way, we find, in the case of the Group 91 of 
246 pupils in Grade 9 and the Group 9II of 192 pupils in 
Grade 9, that : 

Composite L — Composite K = .84a! ai and l.OOai an* 

Composite M — Composite L = 1.44ai bi and .Sooi wi* 

Oi 01 = 1.19ai on or .38ai on, according to the pair of com- 
I)osites used. If we take the most remote pair which in- 
clude all the data, M and K, we have 2.28ai 01 = 1.55a! on, 
whereby ai w = .68a, ou* 

In the same way, we find, with Group 5| and Group 91, 
that: 

Composite J — Composite I = 2.10a! sj and 1.94a! 01. 

Composite K — Composite J == .72ai 5j and l.OOa, »i. 
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Oi Si — .92o, 01 or I. 5 I 01 01 according to the pair of com- 
posites used. If we use K and I, which include all the data, 
we have 2.82ai oi = 3.02oi oo whereby oi si = l.OToi oi* 

For precise work in scale construction, the groups 
should be large and close enough together to have a consid- 
erable overlapping. The measurement of the Oi of any one 
group in terms of the a, of any other group may then be de- 
termined with as small an error as is desired. 

Our groups are obviously not large enough, since there 
are so great differences between the estimates of the com- 


TABLE 113. 

Thh Intellectual Difficulty op Composites I to Q. Averages op the 
Determinations of Table 110 and Table 112. 


Composite 

CT| 

Id Oi *1 

Difficulty 

In Cl Qll 

Id 0 , „ 

Id Oi n 

I 

- 1.49 

-3.37 




j 

+ .61 

-1.43 




K 

+ 1.33 

- .35 

0 

f 

-f 



L 


+ .49 

+ 1.10 



M 


+ 1.93 

+ 1.65 



N 



+ 2.75 

-1.07 

-2.13 

0 




+ .05 

- .77 

P 




+ .66 

- .17 

Q 




+ 2.21 

+ .85 


parative variabilities according to the composites which we 
use. There is particular risk in using the estimates of com- 
parative variabilities in different groups which depend 
upon a composite that is very easy or one that is very hard 
for the group. In the case of the very easy composites 
carelessness may play a part that affects the results. In 
the case of the composites which are very difficult for a 
group, lack of effort and persistence and interest may be a 
disturbing factor; and it is possible that, in spite of care 
taken to give what seemed to be abundant time, certain in- 
dividuals may not have exhausted their abilities for lack of 
sufficient time. The eccentricity of the results with Com- 
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posite M in Group 91 may be due to this fact. In general, 
Group 91 was superior to Group 9II and the reversal to 
notable inferiority with Composite M may be explainable 
by the fact that this was the hardest composite taken. It 
was not truly the last in point of time, since all the C were 
done in one division of the examination, all the A’s in an- 
other division of it, ail the V’s in another division of it, 
and all the D^s in another division of it. 

In cases where the material is not notably richer than 
this of ours and in cases where the groups are spaced so 
far apart that there is little or no overlapping, valuable aid 
may be derived from a general consideration of the com- 
parative variability of groups similar in school grade or 
other indication of intellect to the particular groups which 
are used in scaling the difficulty of the composite tasks. 
Moreover, facts concerning the comparative variability of 
grade populations are valuable as a check on even the best 
determinations made by using two or more composites with 
two or more groups. Consequently we have made a rather 
exhaustive study of the variability of grade populations 
from 6 through 13, using all the data that we could discover 
which had sufficiently large populations to make the deter- 
minations of variability reasonably precise. 

In order to discover the relative variability of different 
grade populations from 6 through 13, if each individual 
were measured in truly equal units, we may proceed in 
either one of two ways : 

We may argue after the fashion of the argument in Ap- 
pendix III that inequalities in the face- value units will neu- 
tralize each other so that the general average result from 
many tests, each with its own sort of inequality, will be near 
the truth. In this case, we simply take the sigmas by the 
original scoring for these different grades and get their 
general drift. Dr. Bregman has done this for all the ma- 
terial available with populations large enough to give re- 
liable sampling of the grades. The results are shown in 
Table 115 and in more detail in Table 114. 
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V 

The second method is to transmute the face-value mea- 
sures for such tests as Army Alpha, National A, Otis Ad- 
vanced, etc., into terms of equal units before computing the 
sigmas. The results of the investigations reported in 
Chapter VII enable us to do this, since in that chapter we 
determined the value in equal units of each interval of the 

TABLE 114. 

Data foe Computing BsiiATivis Vakiabilities of DiPFBaiENT Grades in Intel- 
lect; AND FOB Computing Distances between Medians 
of Different Grades in Intellect. 

Original median refers to the median by the standard method of scoring; 
corrected median refers to the median by a scale in equal units; original a 
refers to the mean square deviation by the standard method of scoring; cor- 
rected a refers to the mean square deviation by a scale in equal units. 


Grade 

Number 

Median 

Original Corrected 

Original 

a 

Corrected 

Army Alpha* 






6 

281 

54.9 

55.6 

18.4 

19.1 

9 

1721 

97.94 

97.94 

24.0 

24.2 

10 

1223 




24.0 

11 

977 




23.8 

12 

1387 

125.39 

125.39 

24.24 

24.8 

12 

766 

128.04 

128.04 

24.13 

24.9 

Coll. 1 

2545 

128.50 

128.50 

28.20 

29.2 

1 

400 

157.8 

158.5 

19.99 

23.3 

Army Examination A* 





6 

742 

139.8 

139.8 

36.9 

38.94 

7 

685 

158.6 

158.6 

39.2 

40.90 

8 

630 

186.1 

186.1 

43.04 

43.39 

9 

311 

204.36 

204.36 

45.89 

45.53 

12 

53 

276 

274 

36 

36 

Coll. 1 

701 

267.33 

265.33 

40.63 

39.25 

National A** 






6 

1668 

111.9 

111.9 

22.8 

21.8 

9 

494 

141.75 

140.85 

16.8 

16.5 


* All computations exact. 
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TABL.E 114 

(Continued). 



Grade 

Number 

Median 

Original Corrected 

Original 

a 

Corrected 

Otis* 






6 

5952 

86.8 

87.2 

24.3 


7 

3896 

96.98 

97.1 

24.4 


8 

4598 

111.93 

111.4 

25.08 


9 

3627 

125.04 

123.8 

24.62 


12 

1226 

151.83 

149.7 

24.06 


Haggerty* 






6 

916 

91.4 

91.3 

20.4 

20.7 

7 

737 

105.07 

105.2 

20.2 


8 

689 

113.9 

113.7 

19.46 


9 

473 

113.7 

113.5 

17.5 

19.54 

9 

1995 

116.5 

116.4 

18.2 

23.25 

12 

668 

135.83 

139.3 

15.31 

22.4 

f. E, B. Form 

A** 





6 

379 

83.9 

81.0 

32.41 


9 

3231 

173.4 

173.4 

42.9 


10 

1935 

191.1 

191.1 

40.3 


11 

1533 

202.6 

202.6 

42.4 


12 

972 

219.81 

219.8 

44.99 


12 

1666 

227.79 

227.5 

45.85 


/. E. B. Form 

B** 





10 

1656 

209.0 


43.55 


11 

1453 

219.7 


44.0 


12 

1207 

229.9 


44.7 


Terman Group 

Test*** 





9 

1438 

102.16 

102.16 

32.0 


12 

4886 

144.55 

142.55 

32.61 



* All computations exact. 

*♦ The sig'znas in equal units will vary inappreciably from the sigmas by the 
original scale and are not computed. 

*•* effect of inequalities in the units will be almost identical for Grade 
9 and for Grade 12; hence the relative values of the sigmas will not be inffu- 
enced thereby. Consequently the sigmas for values in equal units have not been 
computed. 
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original scale for Army Alpha, National, Otis, Haggerty, 
Army a, Terman Group Test, and several others. We have 
made these computations with results as shown in Table 
114. 


TABLE 114 (Continued). 


Grade 

Number 

Median 

Original Ciorrected 

Original 

a 

Corrected 

Brown University* 





12 

3333 

45.69 

46.2 

11.59 


CoU. 1 

2118 

56.62 

56.3 

11.11 


Myers Mental Measure** 





6 

724 

46-3 

46.3 

13.1 

12.6 

7 

696 

49.61 

49.8 

14.65 


8 

950 

54.15 

54.55 

13.72 


9 

311 

57.1 

57.5 

13.05 

13.75 

Pintner Non- Language** 





6 

1237 

316.7 

313.7 

86.7 

86.5 

7 

755 

339.0 

339.0 

73.18 


8 

530 

379.6 

381.6 

73.24 


9 

258 

400.6 

403.0 

75.0 

78.5 

Pressey Cross-Out*** 





6 

1057 

51.18 


10.30 


7 

998 

56.10 


10.30 


8 

725 

63.12 


10.0 


9 

303 

72.5 


10.0 


Trahue Completion*** 





6 

1454 

21.8 


5.5 


7 

1456 

25.39 


5.67 


8 

1740 

27.61 


6.29 


9 

273 

30.05 


5.9 



* The inequalities of units in the scale are such as balance one another and 
leave the relative values of the sigmas by the original units undisturbed^ Con* 
sequently new values are not computed. 

** The sigmas according to a scale with equal units are computed by iinding 
^ the distance required to exclude 15.87% at each extreme. 

*** Scores in equal units have not been determined. 
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The data which we have used to measure comparative 
variabilities are the same as those which will he used 
later to measure the differences between the medians of 
various grade groups in intellect. We present them in 
Table 114 classified according to the examination used. 
In connection with each examination we record the results 
for Grades 6, 9, 12, and 13 (or first year of college) and 
occasionally for other grades or groups. We report the 
number of individuals ; the median score, taking the 

TABLE 114 (Concluded). 


Median a 

Grade Number Original Corrected Original Corrected 


Illinois Exammation* 



6 

588 

75.52 

17.01 


9 

380 

101.4 

18.5 

Thorndike Intelligence Examination^ Part I* 



12 

1527 

91.4 

18.1 

CoU. 

1 

166 

101.7 

17.6 

ti 

1 

466 

108.4 

17.0 

c t 

1 

319 

107.1 

18.5 



Weighted average 



(i 

1 

(weights 1, 2 and 

106.5 

17.7 


2 ) 


* Scores in equal units have not been determined. 


units at their face value; the median score in a scale 
with equal units; the mean square deviation, taking the 
units at their face value ; the mean square deviation, 
using a scale with equal units. In the latter case the 
sigmas have been computed exactly, where it was possible, 
but in many cases we have had to resort to approximations. 
In cases where the scale with equal units was so closely 
similar to the original scale that little, if any, difference 
would be made in the mean square deviation, we have used 
the original figures. Notes are appended to Table 114 de- 
scriptive of what was done in this regard in each case. 
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From the facts of Table 114 are computed the ratios of 
Table 115. 

Prom the facts of Table 115 we may conclude that the 
forces of selection and gradation which determine the vari- 
ability of grade populations result in a slight increase from 
Grade 6 to 9, which we may estimate as 4 percent (giving 
twice as much weight to the results from equal-unit scaling 


TABLE 115. 

The Belative VABiABiiiiTY OP Dipperbnt G&ade Populations. 


Examination 

Using the Original 
Scale Units 

0i« <Tiis Uiu 

Ois <Ti« CTia 

Out 

0112 

Using Seales with Equal 
Units 

01f 0112 0112 0IIt 

01t 0f2 012 0112 

Army Alpha 

.77 

1.01 

1.00 

1.00 

.79 

1.03 

1.09 

1.06 

Army a 

.80 

.78% 

.88% 

1.13 

.85% 

.79 

.86 

1.09 

National A 

1.30 




1.32 




Otis Adv. 

.99 

.98 



.99 

.98 



Haggerty 

1.13 

.85 



.93 

1.00 



I.E.R. Sel. Gen. 

.76 

1.06 



.76 

1.06 



Terman Group 


1.02 




1.02 



Brown Univ. 




.96 




.96 

Myers Mental 

1.00 




.92 




Pintner 

1.16 




1.10 




Th. Part I N 




.97 





Trabue Comp. 

.93 








Illinois 

.92 








Pressey 

1.03 








Median 

.99 

.99% 

.94 

.98% 

.92% 

1.01 

.97% 

1.06 

Average 

.99 

.95 

.94 

1.01% 

.96 

.98 

.97% 

1.04 


as to those from the original scores). From 9 to 12 there 

is little or no change. The medians for the ratios aver- 

Ote 

age l.OOj. The .78^ and .79 of Army a which make the aver- 
ages lower (.95 and .98) are from a very small group of 53, 
which should be given very little weight. This group was 
used because it enriched somewhat our very scanty ma- 
terial on the — ^ comparison. We may then estimate the 

Oil2 


22 
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variabilities of Grades 6, 9, and 12 as 96, 100, and 100. Com- 
paring Grade 13 with both Grade 9 and Grade 12, we find 

for the original-scale units a median — — of .98 and 

<j,i2 or Ota 

an average of .99; for the scales in equal units, there is a 
median of 1.06 and an average of 1.01. The best estimate, 
in view of the fact that the .88^ and .86 by Army a deserve 
less weight than the other determinations, seems to be 
about 102. We then have 96, 100, 100, and 102 as the rela- 
tive variabilities of Grades 6, 9, 12, and 13. 

These general facts may be used to correct the eccentric 
and unreliable determinations from the composites them- 
selves (see page 304 f.). The use of the entire stretch of 
overlapping gave oisi 1.07ai»i, for our particular group, 
but in general onn may be expected to be about .96ai». We 
know of no facts which make it probable that our groups 5^ 
and 91 differ from Grades and 9 in general in such a way 

as to make a variation of --- up from .96, any more prob- 

0|9 

able than a variation down. The scientific procedure would 
be to apply the same examinations to these two particular 
groups, and compute the variabilities in units of known 
value, but this was not practicable. The best thing to be 
done in the circumstances is to attach some reasonable 
weights to the two lines of evidence, and so obtain a work- 
ing estimate. Giving the general facts about Grades 6 and 
9 a weight of 4, and the particular facts from the composites 

used in both groups a weight of 1, the ratio is .98, 

Oioi 

The next matter to be cleared up is the comparative 

variability of 91 and 9II. was 1,19 by L-K and ,38 by 

Oioii 

M-L. We shall disregard these determinations entirely 
and treat the variability of 91 as equal to that of 9II, for 
the following reason. These two groups were constituted 
by a division of all the pupils in Grade 9 in a certain school 
at random, so far as is known. There is nothing in their 
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BTunmation scores to show that one is more variable than 
the other. The L-K and M-L determinations are enor- 
mously at variance, and so deserve very little weight. 

Between 9 and 13 there is no overlapping, so that the 
general facts of grade variability are the only means of 
estimate. As has been stated, our group 13 is a group of 
candidates for college entrance, not of actual freshmen. 
They were, however, candidates already selected by certain 
tests and were of intellect comparable to the freshman 
groups reported in Table 114, differing probably toward 
less variability rather than toward more, if they differed at 

all in this respect. 1.02 or 1.00 is then suitable as the 

0|9 

ratio, so far as is known. 

The last comparison to be considered is of group 13 and 
group 17. The determinations from the composites taken 

in common were: ^*** = 1.21 or .99 or .66, with a median of 

Out 

.99 and an average of .95. The use of the widest stretch 
between composites gave .91. The .66 and .91 and .95 are 
probably too low, inasmuch as all depend on the + 2.21 for 
composite Q in the 13 group. This is the most unreliable 
of the eight determinations, and is probably too high. The 
difference between the general level of ability of group 17 
and that of group 13 is 1.06 by composite N, .82 by O, .83 by 
P, and 1.56 by Q. The median .99 is the most probable esti- 
mate from the composites used in both groups. The gen- 
eral drift of the facts for Grades 6, 9, 12, and 13 gives the 
expectation that the variability in Grade 17 will be some- 
what but not much higher than that in Grades 12 or 13, per- 
haps 1.04 or 1.05 times the variability of Grade 9, ghdng a 

ratio for^”® of about .95. 

Oilj 

The records for Kxamination a with 136 college students 
of Grades 14, 15, and 16, and with 27 graduate students 
show, however, decreases in variability much below that of 
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Grade 9, making well above 1.05. So the general con- 


Out 


siderations can hardly be used to favor change from 1.00 in 

13 17 

either direction. On the whole .99 for or 1.01 for ^ 


is fairly well justified by both methods. 

The values recommended for turning the various Oi’s 
into Oia's are then: 


(Tioi CTidi: 


= .98 


Oi9i — Oiaii 


Oils 

Oiai or (Jioii 


= 1.02 


Out 

0191 or 0,911 


= 1.03 


Nothing in the particular comparisons from the composites 
themselves is inconsistent with these estimates. What has 
been done is to use general considerations to locate ratios 
within the limits of those which were reasonable in view of 
the particular comparisons. Using them the measures of 
Table 113 become those of Table 116. 


EXPRESSING THE MEASURES OF DIFFICULTY AS DISTANCES FROM A 
COMMON POINT OF REFERENCE 

The differences in difficulty of composites I, J, and K 
plus and minus from the median of group 5^ may be ex- 
pressed as differences from the median of group 9T, by find- 
ing the differences between the difficulty for the median of 
group 5| and the difficulty for the median of group 91. This 
may be found by using the composite tasks which were used 
with both groups. Thus composite I is, by Table 116, 1.52oifl 
easier than the task which just 50% of group 5^ can master 
and 3.37019 easier than the task which just 50% of group 91 
can master. By this determination, the difficulty of the 
median task for 5i is 1.85 less than the difficulty of the 
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median task for 91. Using the facts of Table 116 for com- 
posites J and K in the same manner, gives 2.O5oi0 and 
I.7I019. The average is 1.87cr,o ; the median is 1.85oi0. 

In the same manner, K is .35oie easier than the task 
which jnst 50% of group 91 can master and .lOoie harder 
than the task which just 50% of group 9II can master. By 
this determination, the difficulty of the median task for 91 
is .45019 greater than the difficulty of the median task for 
9II. Using the facts for composites L and M gives .6I01Q 
greater and .28019 less. The average of the three determi- 
nations is .260,0; the median is .450,9. 

Composite N is 2.750,9 harder than the task at which 
50% of 9II succeed, and 1.050, g easier than the task at which 
50% of group 13 succeed. So the difficulty of the median 
task for group 13 is 3.80,9 greater than that of the median 
task for 9II. 

Using N, O, P, and Q in similar manner, the difficulty of 
the median task for Group 17 is found to be 1.020,9 or 
.800,9, or .810,9, or 1.34^0,9 greater than the difficulty of the 
median task for Group 13. The average is .99^0,9; the 
median is .920,9. 

Relating the difficulty of the median task for each group 
to the difficulty of the median task for a group half-way 
between 91 and 9II, we have : 


The median for 5^ 
The median for 91 
The median for 911 
The median for 13 
The median for 17 


— Median 91 + 9II 

— Median 91 -f- 9II 

— Median 91 H- 9II 

— Median 91 -f- 9II 

— Median 91 + 911 


Computed 

Computed 

by average 

by medians 

— 1.74 

— 1.621 

+ .13 

+ *221 

— .13 

— .221 

+ 3.80 

+ 3.80 

+ 4.80 

+ 4.72 


The reasonableness of these estimates may be checked 
by the facts for the difference between the median scores 
in Grade 5^ and Grade 9 and Grade 13 in intelligence ex- 
aminations in general, expressed in terms of the variability 
of Grade 9, or in some other unit of measure. 
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We have collected the available facts concerning the 
median scores of Grade 6, Grade 9, Grade 12, and the first- 
year of college, in Army Alpha, Army Examination a, Na- 
tional A, Otis Advanced, Haggerty, 1. E. B. Sel. Bel. Gen. 
Org., Terman Group, the Brown University Examination, 
the Myers Mental Measure, the Pintner Non-Language 
Test, the Trabue Completion, the Illinois Examination, and 
the Pressey Cross-Out Test. They are reported in Table 
114 (on pages 307 to 310, inclusive). For all save the last 
three, we have computed what the differences between the 
medians in question are by a scale of equal units. The re- 
sults, both by the original scale and by the scale with equal 
units, are shown in Table 117. 

TABLE 116. 

The Intellectual DiFncuLTT of Composite Tasks I to Q in Teems op Oi*. 


Diffieulty 


Composite 

By 5% 

By 91 

By 9II 

By 13 

By 17 

I 

-1.46 

-3.37 




j 

+ .60 

-1.43 




K 

+ 1.30 

- .35 

+ .10 



L 


+ .49 

+ 1.10 



M 


+ 1.93 

+ 1.65 



N 



+ 2.75 

-1.09 

-2.19 

O 




+ .05 

- .79 

P 




+ .67 

- .17% 

Q 




+ 2.25 

+ .87% 


The variabilities used in computing Table 117 are, of 
course, the variabilities of the respective groups in the 
ability measured by the particular instrument used, such as 
Army Alpha or National A. (a is Oaiph* or ONationai ; 


Mfl . DlA.ipha9 D^AIpIiae 

IS or 

Ua OAIpbaS 


mNat.t) ®[lNat.e 
<JNat.» 


or the like ; and will be smaller than 


D3cAVD9 

OcAVD 

larger than 
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OAiirti* should be treated just as we treated Ot j ’s. We have 
to estimate Otg from OAipha® or ONat.g or Ooti.e- This has to be 
done rather crudely since neither the self-correlations of 
most of these tests, nor their correlations with any such 
criterion as the score of one' of our long CAVD series, have 
been worked out. The self-correlation of the I. K. B. for 


TABLE 117. 

DlFPEEENCE BETWEEN GRADES IN SCORES ATTAINED IN VARIOUS INTELLIGENCE 

Examinations. 



Using the 
original scores 

Using 

scores 

in Equal Units 


m® — in« 


nio “ nig uiis nip 

m,p-inp 


Off 

Ci 

09 

o» 

09 Ot 

Army Alpha 



1.75 

1.28 

2.50 

Army a 



1.42 


1.38% 

National 



1.76 



Otis Adv. 



1.49 

1.05 


Haggerty 



1.09 

1.21 


I, E. R. 



2.15 

1.20 


Terman 

Brown 

Myers 



.81 

1.26 

.87 

Pintner 

Thorndike 

Trabue 

1.40 

.83% 

1.14 



Illinois 

1.40 





Pressey 

2.13 





Median 

1.40 


1.45% 

1.21 

1.94 

Average 

1.04 


1.45 

1.20 


Median + Average 
2 

1.52 

.83% 

1.45 

1.21 

1.94 .87 


two different forms of the examination taken a year apart 
is .82 for 1,000 boys of Grades 9, 10, and 11, and is .86 for 
489 sixteen-year-old boys in these grades tBailor, ’24, p. 
8]. We have computed the self -correlation of the Terman 
Group Test for 209 cases of high school pupils in Grades 9, 
10, and 11, finding it to be .92. The correlation of the Hag- 
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gerty test against a combined score in Army Alpha, Thnrs- 
tone, Otis, Pressey, and other tests is .89 for a group of 60 
college seniors. This would make the self-correlation about 
.80. The self -correlation in “an entire school,^’ the two 
trials being on the same day, is .90 [Haggerty, ’23, p. 54]. 
From the data given in the Memoirs [’21, pp. 315-17], we 
estimate the self-correlation of Army a as about .80 for a 
grade population. The Otis Self- Administering correlates 
.88 with the Terman Group Test in a group covering Grades 
7 to 12 [Clark, ’25, p. 15]. 

Allowing for the restriction of the range in our groups 
as compared with those reported above, we may expect the 
self-cor relations of these various examinations within one 
grade to vary around a central tendency of about .80 for 
rtjt,. Dividing by V-80, we have, for the data from equal- 
unit scores: 

lUg nis ^ — 1 . 62 oi 9 . 

mi2 nig — 1.35o^io. 

mi8 — ms =2.170,0. 
mia — mi2= .9730,8. 

The same divisor with the data from original scores 
gives: 

nio — nig — 1.700,0. 
nii3 nij2 .9340,0. 

Allowing a weight of 4 to the determinations from scores 
in equal units and a weight of 1 to the determinations from 
the original scores, we have : 

mo — mg =1.640,0. 
nii2 “ nio ^ ^ 1.3o0,o. 
mi3 — mo =2.170,0. 
mio — mi2 = .96^0,0. 

We have two independent estimates of mis — mo, 2.170, * 
by the direct comparison and 2.3140, o by the comparison 
via Grade 12. Allowing equal weight to each gives 2.240, o 
as the combined estimate. 
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The 1.64ai9 for 1119 — m« agrees very well with the ob- 
served results of the Av., — 1 . 74 cti 9 , and the Median, 

— 1 . 62 at 9 ; for 1119 — nis^; and we may reasonably accept 

— 1 . 74 ot 9 or — 1 . 62 ai 9 or the average — 1 . 68019 - We shall 
take the last, and use — 1 . 7 oi 9 as the m 9 — msj difference. 
The observed comparison of 91 and 9II may be taken as it 
stands, there being no relevance of the general facts to it. 
So 91 is .13ot9 or .22^019 above mo and 9II is .13oi9 or 
.22^019 below it. We use + . 2 oi 9 and — . 2 oi 9 . 

The 2.24 oi 9 is much below the observed result of 3 . 6 O 019 
for our Group 13 — Group 9 ; and, since this 3 . 6 O 019 depends 
upon the single determination by Composite N, it is wise to 
consider possible amendments of it in view of the general 
facts. 

The following additional facts will help in the decision. 
The individuals of Groups 13 and 17 were tested with half 
of the Composite M and with D4^, which is only a little 
harder than D4. We can infer approximately what the 
percent of successes with Composite M would have been, 
if it had all been given, by allowance for the missing C 
(Comi)letioii M) and for the replacement of D4 by D4^. 

In the case of the 189 individuals of Group 13 there 
were four who might perhaps have failed to have 20 or 
more right out of 40 in Composite M if they had been tested 
with it. By our estimates two probably would have so 
failed- This gives 1.06% or 2 . 34 crti 3 below the median dif- 
ficulty for Group 13. This, in terms of cr,ia would be 2.75 ; 
in terms of 019 it would be 2.70. This would make the 9 
median 4 . 5 ai 9 below the 13 median. 

Among the 240 of Group 17 there was no individual 
who would not have had 20 or more right if he had been 
tested with all the 40 tasks. There were some who probably 
would have had only 22, 23, 24, or 25 right. By our esti- 
mates 

2 would have scored 22. 

1 “ “ “ 23 . 
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The level of Composite M is then probably more than 
— SotiT below the median of Group 17, but not much more 
than that. A reasonable placement would be — 3 . 2 oti 7 * 
This in terms of out would be 3.64; in terms of 019 it would 
be 3.53. This would make the 9 median 5.32o,e below the 17 
median or about 4.3 oiq below the 13 median. 

In view of these additional facts it seems best to con- 
sider that our Group 13 differs more from our Group 9 
than the college freshmen classes of our general survey dif- 
fered from the ninth grades of that survey, and that the 
3.60aio is approximately correct. This means that we are 
treating Groups 13 and 9 as if only about one in ten of the 
latter were equal or superior to the lowest tenth of the 
former in altitude of Intellect CAVD; and this would not, 
in our opinion, seem too small an overlapping to anyone 
who knew the two groups. 

The difference (1.00ai» av. or .92oi9 median) between 
Group 13 and Group 17 is determined from four different 
composites and with a mean square error of only .11. There 
is no reason to alter this in one direction rather than in 
another. 

So we put all the measures of difficulty of Table 116 into 
differences from the difficulty of the task at which 50 per- 
cent of our Group 9 would succeed by the following : 

— 1 . 7 ai 9 for Group 5^ 

+ .20^9 “ “ 91 

— .2a.» ‘‘ “ 9II 

+ 3.60,9 “ 13 

+ 4.60,9 “ ‘‘ 17 

The results are shown in Table 118. 

The average values, allowing equal weight to each deter- 
mination, are: 

I = — 3 . 2 oi 9 M = + I.8019 

J = — 1.2oi9 N = + 2.5oi9 

K = — .2o,» O = + 3.7 oi9 

li = + .8 oi 9 P = + 4.3oi9 

Q == + 5.7oi9 
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The differences, all in terms of 019, are : 


J-I = 2.0* 
K-J = 1.0‘ 
L-K = 1.0 
M-L = 1.0 


N-M= .7 
0-N = 1.2 
P-0= .6 
Q-P = 1.4 


The measurement of the unreliabilities of these deter- 
minations is beyond our facilities both of time and skill. 
They are doubtless large, perhaps as large as .15. They 
are, however, not as large by far (relative to the differ- 
ences to be measured) as are those of the best forms of the 
Binet. 


TABLE 118. 

The iNTEiiLBCTUAii Difficulty of Tasks I to Q Expressed in Each Case as a 
Difference from the Median Difficulty for Group 9, in Units of ai9* 


Task 

By 5% 

By 91 

Difficulty 

By 9II By 13 

By 17 

I 

-3.16 

-3.17 



J 

- 1.10 

-1.23 



K 

- .40 

- .15 

- .10 


L 


+ .09 

+ .90 


M 


+ 2.13 

+ 1.45 


N 



+ 2.55 + 2.51 

+ 2.41 

O 



+ 3.63 

+ 3.81 

P 



+ 4.27 

+ 4.42% 

Q 



+ 5.85 

+ 5.47% 


THE DIFFICULTY OF COMPOSITES A, B, C, D, E, F, G, AND H® 

As was stated at the beginning of the chapter, the mea- 
surements of these lower levels of difficulty are less secure 
than those of tasks I to Q, since investigations of the form 
of distribution of the various groups used and of their dif- 
ferences in central tendency and variability comparable to 
the investigations in the case of Grades 6 to 13 have not 

* These estimates will be amended bj tbe results from other large groups 
to become 1.9 for J— I and 1.1 for K-J. 

B Composite H contained only 30 single tasks, having no sentence com- 
pletions. 
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been made. The results of such investigations as we have 
made are reported in Appendix VI. 

The basal facts for measuring the differences in diffi- 
culty between A and B, B and C, C and D, and so on, are 
the results of experiments with 180 adult imbeciles of Stan- 
ford Mental Age from about 2i years to 5 years, 100 adults 
of mental age 6 (a few over 84 months), 50 feeble-minded 
comprising all the children graded as Class III in one in- 
stitution for the feeble-minded,® 101 pupils in ungraded 
classes in a large city,^ 163 pupils in Grade 4 (second half), 
311 pupils in Grade 5, and 44 adults, recruits in the United 
States Army. These groups will be referred to in order as : 
im. 3, im. 6, f., sp., 4, 5, and ad. (The use of im. and f. in- 
volves no theory of classification, but is solely for con- 
venience.) 

In groups im. 3, im. 6, f., and sp., the tasks were given 
orally. In groups f. and sp. (and in some cases in group im. 
6), the individual tested was allowed to look at the book- 
let as the questions were asked, and read it if he could. In 
groups 4 and 5, the tasks were all presented in print. The 
comparative difficulty for any given group of oral and 
printed presentation has not been determined. In the com- 
putations of differences between groups in variability and 
central tendency which follow, the assumption is made that 
the pupils in Grades 4B and 5 would do better, but vary 
about as much, if they were tested in the manner used with 
the lower-level groups, as they did when tested with the 
printed booklets. The amount of allowance made will be 
described when the differences of groups below group 4 
from groups 4 and above are computed. 

The percent succeeding for each of the 40-composite 
tasks is reported for such of the groups as were measured 
by that task, in Table 119. Table 119 thus corresponds to 
Table 106. 

• This Glass III corresponds roughly to grade 3 of an ordinary school. 
The chronological ages ranged from 9 to 21, only 6 being below 12 and only 
2 over 18. 

»^The distribution of ages reported was: 15 from 13-0 to 13-11, 37 from 
14-0 to 14-11, 39 from 15—0 to 15—11, and 10 from 16-0 to 16-11. 
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Using for the respective groups the forms of distribu- 
tion derived and described in Appendix VI, the difficulty of 
each 40-composite is found in terms of its difference from 
the difficulty of that 40-composite which exactly half of the 
group in question would'have succeeded with, in terms of 
the mean square deviation of the group in question in the 
ability measured by that 40-composite. These measures 
appear in Table 120, which corresponds to Table 107. 


TABLE 119. 

Pekcents Succekdino with Vabious OoBCPOsiTi&s IN Groups iu3, iuC, p, 

SP, 4, 5, ANU AD. 



Groups in Institutions 
for the Feeble-Minded 
imS iin6 f 

Special 

Classes 

sp 

Regular School 
Classes 

4B 5 

Adult 

Becruitfl 

ad 


MA2i^ 
to 5 
n = 180 

MA6 
to 7 
n = 100 

MA7- 
to 10 + 
n = 50 

n=:101 

11 = 163 

11 = 311 

ti 

II 

A 

88.3 







B 

48.3 







C 

12.8 

98.0 






D 

00.6 

73.0 






E 


45.0 

96.0 

98.0 




F 


14.0 

94.0 

90.0 

100.0 


100.0 

G 


03.0 

66.0 

88.1 

98.8 


100.0 

H 



68.0 

67.3 

91.4 

97.7 

97.7 

I 



06.0 

34.7 

35.6 

63.3 

70.5 

J 





03.1 

13.2 

56.8 

K 





00.0 

00.3 

47.7 


ESTIMATING O, FROM Ot, 

By moans of determinations of rtit 2 for the various 40- 
composites in the various groups, the measures in units of 
Oa ims) Oc im6) ^nd the like, are transmuted into units of 
Oi im 3 > Oi ime> and the like. The essential facts of these deter- 
minations are shown below. The results appear in Table 
122, which corresponds to Table 113. 

In general, we have measured rtna both by the Spear- 
man formula using two twenties, and by the correlations 
of neighboring forties. To economize time, only one method 
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is used in the case of group im. 3 and group im. 6 and group 
ad.; and only 98 of the 180 individuals are used in group 
im. 3. 

The self-correlation of one random half of a 40-com- 
posite with the other half for 98 of the imbeciles of men- 
tal age 2^ to 5 years was found to be .86 J for A, .77^ for 
B, .86 for C, and .76 for D. The self-correlation of one 
40-composite with another at the same level may then be 

estimated ^by r 4 o = .927 for A, .874 for B, .924 

for C, and .864 for D. 


TABLE 120. 

The DirFicui/TT or Composites A to K, in Various Groups Expressed as a 
Deviation from the Difficulty foe the Median of that Group, in 
Teems of the a or that Group in the Abiuty Measured by 
Success with the Compobitk in Question, 


Group 

n 

im 3 
180 

im 6 
100 

f 

50 

sp 

101 

4 

163 

5 

311 

ad 

44 

A 

--1.68 







B 

4- .05 







C 

+ 1.13 

--1.90 






D 

+ 1.83 

- .45 






E 


+ .29 

-1.33 

-2.61 




F 


4 1.25 

- 1.25 

-2.31 

<-3.10 



G 


4-2.08 

- .33 

- 1..'>4 

-2.20 


<-3.10 

H 



- .41 

- .44 

-1.37 

-2,00 

1 

to 

b 

o 

I 



+ 1.17 

+ .36 

+ .37 

- .34 

-1.00 

J 





+ 1.87 

+ 1.12 

- .35 

K 





> +3.10 

+ 2.75 

+ .06 


Dividing the entries under im 3 in Tables 120 by ^.927f 
V*874, V-924, and V*864, respectively, we obtain values in 
terms of o, ,„3 from the values for ims, <Tb ims, etc. They 
are : — 1.74, + .05, + 1.18, and + 1.97, as shown in Table 
122 . 

The inter-correlations of the 40-composites C, D, E, F, 
and G in the case of the 100 adults of mental age 6 were as 
shown in Table 121. The correlations with neighboring 
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composites were .685 for A, .703 for B, .725 for C, .769 for 
D, and .809 for E. We add .03 to obtain estimated rtit 2 ’s. 

Dividing the entries in the im 6 column of Table 120 by 
V.715, V-733, V-755, V.799, and V-839, respectively, we 
obtain values in terms of oj ,„o from the values for Oc iiiie» 
Od imsj Ob Im6> ctc. Thcy are : — 2.25, — .53, + -33, + 1.40, 
and + 2.27 as shown in Table 122. 

The self-cor relation of one random half of a 40-compos- 
ite with the other half for group f (the 50 feeble-mirsded in 
class 3) was found to be .638 for E, .809 for F, .638 for Q-, 
.876 for H, and .588 for I. The self-correlation of one 40- 


TABLOE 121. 

Haw iNTERCORREUtTIONS OP CJOMPOSITBS C, D, E, F AND G IN THE CASE OP 100 
Individuals Chronologically Sixteen or Over, and Mentally Six. 



D 

E 

F 

O 

c 

.685 

.685 

.588 

.426 

D 


.721 

.6.^8 

.426 

£ 



.729 

.509 

F 




.809 


composite with another at the same level of difficulty is thus 
^by r^o = f -779 for E, .894 for F, .779 for G, .934 for 
H, and .741 for 1. 

The inter-correlations of the 40-composites E, F, G, H, 
and I for group f, were : E with F — .59, F with G = .81, G 
with 11 ~ .88, and H with I — .81. The correlations with 
neighboring composites are thus .59 for E, .70 for F, .84J 
for G, .841 for H, and .81 for I. Adding .03 as an allowance 
for remoteness gives .62, .73, .874, .871, -84. 

Allowing equal weight to these two determinations, the 
values of rtus are, respectively, .70, .81, .83, .90, and .79. 
Dividing the entries in column f in Table 120 by V*70, V*81, 
V.83, V*90, and V*79, respectively, we obtain values in 
terms of an from the values of OEtj Oro Oau <Jh«» and ott- 
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They are — 1.59, — 1.39, — .36, — .43, and 1.32, as shown 
in Table 122. 

In group sp (the 101 pupils in special classes) the inter- 
correlations of neighboring composites were: F with G = 
.62, G with H — .77^, H with I = . 86 . Adding .03 allowance 
for remoteness, rtit 2 is .65 for F, .73 for G, .82 for H, and 
.89 for I. 

The self-correlation of one random half of a 40-com- 
XK)site with the other half in group sp is .73 for F, .54 for G, 
.64 for H, and .82 for I. The correlation of a 40-composite 
with another of equal diificulty, that is, rtit 2 , may by these 

facts be estimated (^y ^ 40 = as .844 for F, .701 

for G, .780 for H, and .901 for 1. 

Giving equal weight to these two determinations, we 
have, as values of rtit 2 j -75, .71^, .80, and .89^ for F, G, H, 
and I in group sp. Dividing the entries in the sp column 

of Table 120 by V-75, V*715, V-80, and respec- 

tively, we have values in terms of 0 , ,p from the values of 
Gss BP, Gf bp, etc. They are — 2.67, — 1.82, — .49, and -h as 
entered in Table 122. ^ 

In group 4 (the 163 cases of Grade 4B) the int''^ 
lations vrere : G with H = .83^ ; H with 1 = .86 ; I w^ h j = 
.63 ; J with K = .47. Adding .03 as allowance for remote- 
ness, rtita is . 86 ^ for G, .88 for H, .77^ for I, and .58 for J. 

In group 4 the self-correlations of one half with the 
other half of each 40-composite were .69 for G, .79 for H, 
.83 for I, and .65 for J. The correlation of a 40-coraposite 
with another of equal difficulty is thus .817 for G, .883 for H, 
.907 for I, and .788 for J. 

Giving equal weight to the two determinations of rtit 2 ) 
we have .84, . 88 , .84, and . 68 f for G, H, I, and J, respec- 
tively. Dividing the entries in the 4 column of Table 120 by 

V- 88 , and V- 68 i, respectively, we have values 

in terms of 014 from the values of gf 4 > Og t? Gh a etc. They 
are — 2.47, — 1.47, +.40, and +2.26, as entered in Table 
122 . 
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The intercorrelations in the case of the 311 pnpils of 
Grade 5 were : H with I ==.77, I with J = .85, J with K = 
.61. The correlations with neighboring composites, ele- 
vated .03 to allow for remoteness, are thus: .80 for H, .84 
for I, .76 for J, and .63 for K. 

The self-eor relations in this group, using 20 elements 
with 20, are : .68, .77, .70, and .51 for H, I, J, and K in order. 
The correlation of one 40-composite with another of equal 
difficulty would then be .81 for F, .87 for I, .82^ for J, and 
.671 for K. 

Allowing equal w^eight to the two determinations of rtit2. 


TABLE 122. 

The IiimcTJETY op Composites A to K in Terms op Otmt , Ot , etc. 



im3 

iiu6 

f 

sp 

4 

5 

ad 

A 

- 1.74 







B 

+ .05 







c 

+ 1.18 

-2.25 






D 

+ 1.97 

-- .53 






E 


+ .33 

- 1.59 





F 


+ 1.40 

-1,39 

-2.67 




a 


H 2,27 

- .36 

-1.82 

-2.47 



H 



- .43 

- .49 

-1.46 

-2.23 

-2.34 

I 



+ 1.32 

+ .38 

+ .40 

- .37 

-1.09 

J 





+ 2-26 

+ 1.26 

- .35 

K 






+ 3.41 

^ .06 


we have ,80J for II, .851 for T, .79 for J, and .65 for K. Di- 
viding the entries in Column 5 of Table 120 by V-805^ 

V-855, V-79, and V-O*’? respectively, we have values in 
terms of 015 from the values for Gu 5, Oi sy Oj 6* Ok 5- They are 
— 2.23, — .37, -4- 1.26, and -j- 3.41, as shown in Table 122. 

In the case of the 44 adults, the intercorrelations of the 
40-coniposites were : G with II == .73, H with I = .65, I with 
J = .96, and J with K = .91. Allowing + .03 for remote- 
ness, we have .78, .73, .831, .951, and .94 as the probable 
correlation of G, II, I, J, K, each with another 40-composite 
of equal difficulty. Dividing the entries in Column ad of 

23 
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Table 120 by V*73, V*835, V*955, and V*94, respectively, 
we have values in terms of Oi .d from the values of go ad> 
Oh ad> Oi adi Oj ad» Rud Ok ad* They are .2.34:, ~~ 1.09, — .35, 
and H- .06, as shown in Table 122. 

EXPRESSING THE Oi OF EACH GROUP IN Tl’lRMS OF O19 

In accordance with the earlier findings, Oiw and Gimi are 
treated as equal. 

The Gi of group 5 (311 pupils in Grade 5) is made com- 
parable with gi 9 by finding the difference in difficulty be- 
tween two tasks in terms of gis and in terms of Gim, which is 
equal to gio- Thus 

K-J = 2,15gib and 1.09 oibi, whereby O15 = .Sloiei. 

J-I = 1.63gib and 1.94 gioi> whereby gi# = 1.19gi9i. 

It is also possible to proceed indirectly by way of Gidj, which 
was found to equal .98ot8. Thus 

K-J = 2.15gi 5 and .72016^ or .TOSgib, whereby Gis = . 33 gi 9 . 
J-I == 1.63gib and 2.10 gi 64 or 2.O6019, whereby 015 = 1.26ai9- 

It is also true in general that the variability of Grade 5 in 
intellect will not be much different from that of Grade 5^. 

If an estimate had to be made from general considera- 
tions, Gi 5 would be expected to be at least .95ai9. AVc assign 
equal weight to .85 (the median of the .51, 1,19, .33, and 
1.26) and to .95; and use .90oi9 as the value of 015. The 
— 2.23, — .37, 1.26, and -{- 3.41 of Table 122 in terms of 

Gi 5 thus become the — 2.01, — .33, -j- 1.13, and + 3.07 of 
Table 123 in terms of gib- 

Next, the gi 4 is put in terms of gib both directly and via 

OiB* 

J-I = 1.86 gi 4 and 1.94 gi 9, whereby gu = 1.04gi9. 

J-I = 1.86 gi 4 and 1.63 gi 5, whereby gi 4 = .88 gj 5 or .79 gi 9. 

I-H = 1.86gi4 and 1.86 g, 6, whereby Oi4 = 1.00gi 6 or .90 gi9. 

Prom these facts, Oi4 is taken to be apiiroximately equal 
to .94Gt6 or .85 gib* The — 2.47, — 1.46, ~t- .40, and ~t“ 2.26 of 
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Table 122 in terms of a 14 thus become — 2.10, — 1.24, + .34, 
and + 1.92 of Table 123 in terms of O 19 . 

Next the oisp (the 101 special class pupils) is put in 
terms of 019 via oic and pia 014 . 

H-G — 1.33<ii9 and I.OI 014 , whereby Oiap= .76 <t, 4 or .65ai«. 
I-H = .87 oibp and 1 . 860 , 4 , whereby o, 8 p = 2 . 140,4 or 1.93oig. 

Nothing is known precisely of the general tendency of 
pupils over 13 in such special-class populations to vary, 
though the expectation would be that the variation would 
be fairly wide, from pupils who really belonged in an insti- 
tution for the feeble-minded to pupils who really belonged 
in a regular Grade 4. Giving equal weight to the three de- 
terminations, OiBp = 1 . 47 oi 9 . Giving equal weight to the 
I-H and the H-G pairs, o,.p = 1.26oi9. We use the latter. 

In a similar manner o,,, o, ims, and o, ims are put in terms 
of 0 , 9 * The essential facts are : 

I-H = 1.75a, f or 1 . 860,8 or 1 . 860,4 or .87ot«p, whereby 

Oif • — I.O 6015 .9oo,9, 

or Oif = 1 . 06 o ,4 or .90 oi 9, 
or oif — .50a, sp or . 630 , 9 - 

H-F = .96o,f or 2.18o,sp, whereby 0 ,,= 2.27oisp or 2 . 860 , 9 . 
We take the median of these four observations, .92^0,9- 

G-F = .87o, imc and 1.03o,t and .85o,ap» whereby 
o, ,m6 1.18o,, or 1.09o,9 or 

o, ,nifl ■ — •9 ^o',bp or 1 . 22 o, 9 . 

F-F = 1.07o, ,m 6 and .20o,i, whereby o, ,^9 — .187o,f or .17o,9. 

Since group im 6 contains only individuals of Stanford 
Mental Age 6 ,® it may be assumed to be much less variable 
than group f or group im3, or any other group used here. 
The average of the three determinations (l.OOoip, 1.22oi», 
and . 170 , 9 ) which is .83o,9, is used, giving in terms of 0,9 
— 1.87, — .44, -f- .27, + 1.16, and + 1.88 as the entries in 
Table 123. 

Bpltus two individuals of mental age 7. 
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I>-C — ,79cr, in,8 or 1.72o, i„6, whereby 

Oi imS == 2,180, ,u ,6 or 

Using o, lin3 1.81 0,e, the entries for Table 123 are — 3.15, 

+ .09, + 2.14, and + 3.57. 

The facts for the adult group are : 

K“J = .41a, ud or 1.090,9 whereby 0iad = 2.670,9. 

J-I == -740, ^d or 1.940,9 whereby 0,ad = 2.62o,9. 

I-H == 1.25a,ad or 1.860,5 or 1.860,4 or .870, ,p or 1.750,,. 

By these four indirect computations, 0,^^ = 1.490,5 or 1.340,9. 

o,,d ~ — 1.490,4 or 1.270,9. 
0,ad .700, sp or .880,9. 
0 ,ad — “ 1.4001, or 1.300,9. 


TABLE 123. 

The DimcuiiTY op Composites A to K, in Terms op o„. 


im3 im6 f sp. 


A 

-3.16 




B 

+ .09 




C 

+ 2.14 

-1.87 



D 

+ 3.57 

- .44 



£ 


+ ,27 

-1.47 


F 


■\ 1.16 

-1.29 

-3.36 

a 


+ 1.88 

- ,33 

-2.29 

H 



- .40 

- .62 

I 



1 1.22 

4- .48 


J 

K 


4 5 ad. 5% 91 


- 2.10 

-1.24 

-2.01 

-5.12 



■t .34 

- .33 

-2.39 

-1.52 

-3,37 

+ 1.92 

+ 1.13 

- .77 

+ .62 

-3.43 


+ 3.07 

+ .13 

H 1.36 

- .35 


We weight each of the direct comparisons as equal to 
the median of the four indirect comparisons and thus by 
averaging, have, as the estimate used, 0, ad — 2.19a, ». This 
value is not unreasonable, since the group of adults in- 
cluded men of schooling all the way from Grade 3 to Grade 
12. The entries for Table 123, derived from Column ad in 
Table 122, are then : — 5.12, — 2.39, — .77, and H- .13. 
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EXPBESSING THE MBASUEES OF DIFFICULTY AS DISTANCES FBOM A 
COMMON POINT OF BEFEBENCE 

The measures of Table 123 are in every case distances 
from the median difficulty for the group in question. We 
shall express each (in Table 124) as a distance from the 
median difficulty for Grade 91 + IT, in terms of <j,e as a unit, 
by estimating the distance of the median difficulty for 
group 51 from the median difficulty for group 91 + II, and 
similarly for the median difficulty for each of groups ad, 5, 
4, sp, f, im6, and im3. The essential facts and procedures 
are stated below. We use M91 + n to denote the difficulty of 
the task which exactly 50% of the Grade 9 group will suc- 
ceed with, Mr, to denote the difficulty of the task which ex- 
actly 50% of the group 5 will succeed with, and similarly 
for M,™3, M,„6, Mt, M„p, M4, Mad, Msj, Mgj, and M»n. 

K = JNfsj + 1.36 g,9. 

== M5 + 3.07ctio, whence Ms^ — Mg = 1.71g,9. 

J = Mgj + .620i9 

= Mg +1 .13ai9, whence Mgj — Mg = .51ai». 

I - — — l.u2flrn>. 

— Mg ~~~ •33oio, whence IVIgj — ^Ig ■ ■ 1.19oio* 

J = Mg^ -f- .G20(g. 

= M4 + 1.92o,o, whence Mgj — M4 = 1.30ai». 

I = Mgj — 1.520,9. 

= M4 + .340,0, whence Mgj — M4 = l.SGu,®. 

J = Mg -j- 1.130,0. 

== M4 + 1.920,9, whence Mo, — M4 = .790,9. 

I = Mg + .330,0. 

— M4 + .340,9, whence Mr — M 4= .67o,0. 

H = Mg — 2.010,9. 

= M4 — 1.240,9, wffience Mg — M4 = .770,9. 

In view of the above, we take : 

.700,9 as the difference between M4 and Mg, 

.900,9 as the difference betw’^een Mg and Mri, and 
1.600,9 as the difference between M4 and M51. 
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These figures fit everything well, except the determina- 
tion of Mg, — M5 by K ; and this we believe deserves less 
weight than the others. 

I = — 1.52<Tte 

== Mgp .48oi«, whence Mgj — Mgp = 2.00 cth> and 

M4 — Mep = 2.000,9 — 1-600,9, or .400,9. 

I = Mo — .330,9 

= M»p -f- .480,9, whence Mo — M^p = .810,9 and 

M 4 — Map= .81,9 .700,9, or .11019. 

H = Mo — 2 . 010,9 

= Map — .620,9, whence Mo — Mgp = 1.390,9 and 

M!4 ■ — 1.390,9 .700,9, or .69(9a 

I = M 4 + .340,9 

= Map -f- .480,9, whence M4 — M^p = — .140,9. 

H = M4 —1.240,9 

= M,p — .62019, whence M4 — Map = — .620,9. 

G = M4 — 2.100,9 

= Map — 2.290,9, whence M4 — M^p == .190,9. 

Taking these six differences at their face value, there is 
a median difference of .100,9 and an average difference of 
.110,9. We have to allow for the fact that presentation was 
oral to group sp. Lacking experimental evidence, this al- 
lowance is arbitrary. We allow .270,9, making the special 
class .400,9 below group 4. 

I — 1.<j20,9 

— M, + 1.22 o, 8, whence Moj — M, = 2.74o,9 and 

Map Mt== .74019. 

I — Mg — .330,9 

== M, -j- 1.220,9, whence Mg — M, = l..'550,9 and 

Map Mf= .450,9. 

n = Mo —2.010,9 

= M( — .400,9, whence Mg — M, = 1.610,9 and 

Map — M, = .510,9. 

I = M4 -j- .340,9 

= Mt + 1 . 220 , 9 , whence M 4 — M* = . 880,9 and 

Map — Mf= .480,9. 
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H = M4 —1.24a, 9 

= Mt — .40aie, whence M4 — M* = .84a,9 and 

Map — Mt = .44ai». 

G = M4 — 2.10o,fl 

= M, — .33ai 9, whence M4 — Mf = 1.77ai9 and 

IMsp IMt — 1.27ai». 

I = Map + . 480,9 

== M, + 1.22a,9, whence Map — Mf = .740,9. 

I = Msp — .62 oi9 

= Mt — .40o,9, whence Map — Mf = .22oi9. 

= Map — 2.29 o,9 

= Mf — .33 oj 9, whence Map — Mf = I.86019. 

= Map — 3 . 36 oi 9 

= Mf — 1.29 oi 9, whence Map — Mf = 2.07oi9. 

The four direct comparisons with the sp ^oup which 
had oral presentation are the most important. Their aver- 
age is I.22019; their median, 1.30oi9. The average of the 
other six is .650,9; their median, 50o,9. We use 1.10o,9, 
which is very close to the result obtained by weighting the 
result from direct comparison 3 and the result from indi- 
rect comparison 1. This puts the f group as 1.50oi» below 
group 4, which is not unreasonable, since these feeble- 
minded individuals were doing approximately the work of 
a regular school grade 3. 

G = Map — 2.29o,fl 

= M,„ e 1.880,9, whence Map — M,„ « = 4.17oi9 and 

Mf Mim 9 3.07o,9. 

F = Map — 3.36 o,9 



Mjn, 0 

-f- 

1 . 160 , 9 , 

whence 

Map- 


> 4.52aj9 and 








C 3.42<Ji9, 

G 

= Mf 

— 

.33ot9 








1 . 880 , 9 , 

whence 

Mf - 

Im 0 

= 2.2101O* 

F 

==Mf 

— 

1.29o,9 






•®llm 6 


1.16o,9, 

whence 

Mf - 

6 

■ 2*430i99 

E 

= Mf 

— 

1.47o,9 







+ 

.27oi9, 

whence 

Mf - 


■ 1.74019. 
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T1i8 8 .V 0 rag 6 of tliG fivG dGlGPiifiiiiatioii s is 2 « 580 io^ the 


median is 

2.45gio. 

We use 



D 

Im 6 

.44oib 





IVTim 8 

+ 3.57019, 

, whence 

Im 6 " 

M„„ 3 = 4.010,0. 

C 

0 

— 1.870.9 





~ Ini 3 

+ 2.1401B) 

whence 

Im 6 

IM.ni 3 ~~ 4.01010. 


We use l.OOojo* 





These determinations of ditTerenccs are obviously far 
less reliable than is desirable, and should some time be 
made precisely. Thej'^ are, however, presumably free from 
constant errors, and the variable errors do not prevent 
them from satisfying one main purpose of relating the mea- 
sures of I, J, K, L, M, N, O, and P to an approximate abso- 
lute zero. We find the differences between lilo and Mr, and 
M 4 ,etc., as follows : 


Mb — Maj = l.TOojo, by the data presented earlier in the 
chapter. 


Mai 

-Ma 

= .900,9 



and Mo — Mg 

= 2.600,9. 

M* 

— M 4 

= .700,9, 

SO 

Mai- 

— M 4 =1.600,9, 







and Mo — M 4 

= 3.300,0. 

Md 

— Map 

= .400,0, 

so 

Mai - 

- Map = 2 . 000 , 0 , 







and Mo — M^p 

= 3.700,9. 

M,p 

-Mr 

= 1 . 100 , 9 , 

so 

Mai- 

- M, = 3.100,9, 







and Mo — M, 

— 4.80019. 

Mr 

M.ns 0 

— 2.500,0, 

so 

Mai - 

~ M,m A = 5.600,9, 







and M» — M„„ 

A ■ — 7.300,9. 

M.m 3 

M.m 3 

= 4.000,9, 

so 

Msi- 

- Mtn, 3 = 9.600,0, 







and Mo — M„p 3 

= 11 . 300 , 9 . 


These facts are used to put all the entries of Table 123 
into differences from the median difficulty of group 9. The 
result is Table 124, which is thus a continuation of Table 
118. 

The adult group is given its location by the following: 

K = Mad+ .IBcifl 

= Mbi — .35ai», whence Mbi — Mad — .48ai9. 
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J — l^ad *77019 

= M91 — 1.43019, whence M91 — Mad = -660,9. 
I = Mad 2.390,9 

= Mfti — 3.370,9, whence M», — M^d = .980,9. 
K = Mad + .130,9 

= Me, + 1.360,9, whence M^d — Mr,i = 1.230i». 
J = Mod — .77019 

= Mbj + .620,0, whence M„d — Mg^ = 1.390,9. 
I - — M^„d — 2.390,9 

== Mgi — 1.520,9, whence Mad — Mg* = .870,o. 


TABLE 124. 

The iNTEiiECTXJAii DipncHETT OE Tasks A to K Expresseo as a Dippebence 
PROU THE Median Dippichett por Gboiip 9, in Units op 0,,. 


im3 un6 f sp. 4 5 ad. 5^ 91 

A - 14.4.1 

B - 11.21 

C - 9.16 -9.17 

D - 7.73 -7.74 

E - 7.0.3 - 6.27 

P -6.14 -6.09 -7.06 


G 

-5.42 -5.13 

-5.99 

-5.40 





H 

-5,20 

-4.32 

-4.54 

-4.61 

— 5.65 



I 

- 3.58 

-3.22 

-2.96 

-2.93 

-2.92 

-3.32 

-3.17 

J 



-1.38 

-1.47 

-1.30 

-1.08 

-1.23 

K 




•»- .47 

- .41 

- .34 

- .15 


Thus Mjd is on the average .710, o below Mg,, or .510,9 
below Mo and 1.160,9 above Mgj. Since hlg is 1.7O0,o above 
Mgj, this second determination is equivalent to .560,9 below 
Mg. We use the average, .580,9, and so transmute the 

— 5-12, — 2.39, — .77, and + .13 of Table 123, into — 5.65, 

— 2.92, — 1.30, and — .41 in Table 124. 

The data from groups 5^ and 91 are also repeated in 
Table 124. Table 124 then has four determinations of the 
difficulty of K, five determinations of the difficulty of J, 
seven determinations of the difficulty of I, and so on. 
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We first consider whether there is any need to modify 
our earlier estimate of the difficulty of K, J, and I in view 
of the extensive new facts. 

K was — .2oifl from the median of Group 9, hy the aver- 
age of three determinations ( — .34, — .15, and — .10) from 
Groups 5^, 91, and 9II. It is — .51ai9 l>y the adults and 
-f- .47ai» hy Group 5. In view of the great discrepancy, and 
the fact that K is perhaps too hard a task to enlist full 
effort from pupils in Grade 5, we leave the — .2 as the esti- 
mate for K. J was — 1.16 g, 9 from the median of Group 9. 
It is — l.SSoia hy the averages of the determinations from 
4, 5, and ad. Then 1.3a, » is more probable than 1.2aio as its 
value. I was — 3.2ai» by Groups 5^ and 91. It is — 3.12aj9 
by the average of the five new determinations. We leave it 
as — 3.2aio- So we may use — 3.2o»9 for I, — I.Sgiq for J, 
and — .2019 for K- The difference J-1 is then 1.9a, 9 instead 
of 2.0019, and the difference K-J is l.la,,, instead of 1.00,9. 

The difficulty of H is — 4.70,9 or — 4.60,9 (average and 
median); that of G is — 5.40, »; that of F is — 6.4a, » or 
— 6.10,9; the difficulties of E, D, C, B, and A are, in order, 

■ 6. 6^0, 9, 7.70,9, 9.2 o,9 , ~~ H"^Oi9, and 14.4'^0j9. 

The difficulties for L, M, N, O, P, and Q were found (in 
Table 118) to be respectively + .80,9, + 1.80,9, + 2.60, », 
+ 3.7^0, 9, + 4.40,9, and + 5.60, ». 

These measures are all deviations from the difficulty for 
the median of Group 91 + II. Expressed as deviations 
from the difficulty of Task A, they are, in the A, B, C order, 

A = 0, B = 3.2i, C = 5.2i, D = 6.7i, E==7.8, P = 8.2, 
G = 9.0, H = 9.8, I = 11.3, J = 13.1i, K = 14.2^, L = 15.2^, 
M = 16.2i, N = 16.9 J, O = 18.2, P = 18.8i, and Q = 20.0^. 

THE SCALE 

In Chapter X it will be shown that the distance from an 
approximate absolute zero of intellectual difficulty to the 
difficulty of Composite A is about 4.35 times the difference 
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in difficulty between Composite A and Composite C, which 

is 5.28^. 

So the difficulties of A, B, 

c, 

etc., measured from 

an approximate absolute zero are,® 

in 

units of ai»: 

A 

23 

J 

36i or 36.1 

B 

26i 

K 

37i or 37.2 

C 

28i 

L 

38i or 38.2 

D 

29f 

M 

39^ or 39.2 

E 

30| 

N 

40 or 39.9 

F 

31i 

O 

41 or 41.1 

G 

32 

P 

41| or 41.7 

H 

32| 

Q 

43 or 43.1 

I 

34i or 34.2 



The unit ui# may now be given a more realistic defini- 


tion. It is one-twentieth of the difference in difficulty be- 
tween such composites as an adult of approximately mental 
age three can succeed with (in the sense of obtaining 20 or 
more right of the single tasks) and such as only the ablest 
fifth of college graduates can succeed with. It is one-twen- 
tieth of the difference betw^een tasks at which over 999 per 
thousand adults can succeed, and tasks at which only about 
ten per thousand can succeed. It is one-tenth of the differ- 
ence between tasks at which nineteen out of twenty pupils 
in Grade 5 can succeed, and those at which only a fifth of 
college graduates can succeed. It corresponds to about 
average years of mental age from 6 to 12. 

The relative magnitudes from 23 for A to 43 for Q will 
seem preposterous to many critics who will deem it incred- 
ible that the intellects of the top one percent of men should 
be less than twice as ‘‘high” as the intellects of the lower 
ranges of asylum inmates ; or that a child of three has at- 
tained two-thirds of the intellectual altitude which he ever 
will attain. 

The relative magnitudes are far from secure, depending 
so much as they do on the experimental determinations 

0 The hrst column g^ives the measures to the nearest quarter-unit, which 
is even closer than the data justify for tasks A to H. The second column 
gives measures to the nearest tenth of a unit for tasks I to Q. 
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from the im3 group and the expert ratings to be described 
in the next chapter. The 23 for A may conceivably be in 
truth only 13 or 10 or even lower, though we have no evi- 
dence that it is likely to be lower rather than higher. Kven 
if it were as low as 10 or 12, the critics would still find the 
relative magnitudes nearly as preposterous. The difficulty 
is probably due to a confusion of altitude of intellect with 
intellect in toto, a confusion which the Binet and other tests 
seem to have stimulated- Intellect in toto is proportional 
to altitude only if the number of tasks is approximately 
equal at each level of difficulty- The number selected to 
make an examination, such as the Binet, or our OAVD 
series, may be so, but we shall show that the number of 
tasks that are or can be made increases with their difficulty, 
so that what we may call the ‘*area^* of an intellect of alti- 
tude 40 may be, not two times that of an intellect of altitude 
20, but twenty or two hundred or perhaps two thousand 
times it- Any further discussion of these matters may best 
be deferred until after the treatment of the location of zero 
difficulty in Chapter X, and the treatment of the measure- 
ment of width and ^^aroa’^ of intellect in Chapter XIT- 



CHAPTER X 


The Absolute Zero of Intellectual Difficulty 

We may expect the same sort and amount of advantage 
to the scientific study of intellect from a determination of 
its absolute zero as accrues to the study of temperature or 
electrical resistance from the determination of their abso- 
lute zeros. Just as we cannot properly add or subtract or 
average numbers representing degrees of intellectual diffi- 
culty until we know that the units called equal are really 
equal, so we cannot properly make the “times as much” 
judgment, or divide one amount of intellectual difficulty by 
another to form a ratio, until we can state these amounts 
as differences from a true absolute zero meaning just barely 
not any intellectual difficulty. 

Knowledge of the location of zero intellectual difficulty 
not only will put all our measures of difficulty, or altitude 
of intellect, into numbers capable of treatment in ratios, 
but also will put all our measures of what may be called the 
total “surface” or “area” of intellect,^ into numbers 
capable of similar treatment. 

Relations witliin the field of intellect, and relations be- 
tween it and other facts measurable in known units from a 
known absolute zero will then be susceptible of simple, 
straightforward study and presentation as lines in the -f- + 
quadrant of a system of coordinates. In place of our pres- 
ent laborious and somewhat ambiguous determinations by 
correlations and regressions, that such and such an excess 
over or deficiency below the central tendency for a certain 
group in trait Y is related to such and such an excess over 
or deficiency below the central tendency of the same group 
in trait X, we shall have simifie and definite statements that 
y = ax + b, found by plotting the straight line of best fit, 

1 These terms will be defined in Chapter XI. 


S39 
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just as the physicist or engineer does. Where the relation 
is curvilinear and intricate, the gain will be even greater, 
since in such cases the correlation and regression technique 
is especially laborious and subject to ambiguity and mis- 
leading. 

We shall also be stimulated to study psychological re- 
lations over a large part of the range of intellect from near 
zero to near its maximum. The intellects of dogs, cats, 
primates, and men will more easily be made commensurate 
and put in relation to the same fact in the same plot. 


LOCATING ZERO DIFFICULTY BY EXPERIMENT 

By the methods previously described, we can measure 
differences in difficulty in Intellect CAVl) from tasks such 
as only one adult in a thousand can do down to tasks of 
level A (shown in Chapter III) which over nine hundred 
and ninety-nine adults in a thousand can do. Tasks in giv- 
ing the opposites of words or in answering informational 
questions can be measured in respect of their intellectual 
difficulty from an equally high down to an equally low level. 

Consider now such a series of tasks as AA to AAAAA 
below. Adult imbeciles and idiots who cannot do twenty 
out of forty tasks as hard as those of Level A, may succeed 
with twenty out of forty as hard as AA. A hundred dogs or 
cats of specified age and training could be measured as to 
their ability in comiiosites of tasks like AA to AAAA. Am- 
phibians and fishes of specified selection and training could 
be measured as to their ability in composites of tasks like 
AAA to AAAAA.® 


AA 

AAA 

AAAA 

AAAAA 


Responds to the direction “Come here.” 
Can find his way to some very familiar 
place, such as his own sleeping-place. 
Will not try to eat a familiar nasty-tast- 
ing object. 

Will not try to bite off his own toes. 


2 We are, of course, here concerned with the difficulty of tasks learned 
by intellect, not with that of tasks provided for by original nature. 
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A Program of Tasks to Use in Measuring Tasks of Very 
Little Intellectual Difficulty. 

Composite Task. Groups in Which the Percent of Suc- 
cesses Is between 100 and 0. 


I 

Earthworms 


II 

it 


III 

it 

Crabs 

IV 

it 

it 

V 

Fishes 

it 

VI 

it 

it 

VII 

it 

Frogs 

VIII 

a 

it 

IX 

Turtles 

it 

X 

it 

it 

XI 

it 

Ba.ts 

XII 

it 

9 4 

XIII 

Cats 

it 

XIV 

it 

it 

XV 

i t 

Dogs 

XVI 

it 

it 

XVII 

Moukeys 

it 

XVIII 

it 

it 

XIX 

it 

Chimpanzees 

XX 

it 

it 

XXI 

Human adults, A® 

it 

xxri 

it 

it 

XXIII 

it 

Human adults, ! 

XXIV 

it 

it 

XXV 

Human adults, C“ 

it 

XXVI 

it 

it 

A* 

ti 

Human adults, ! 

B* 

it 

it 

c* 

Human adults, E 

it 

D^ 

ti 


s The A, C, and £ groups of human adults are to have means and 
mean square variations in mental months of about 12 ± 4 , 20 ± 4 ^ 30 :±: 4 , 
39 ±: 4 ^ and 43 dt 4 . 

4 These tasks A, C and D are the CAVD composites A, C and D 
or tasks equal to these in intellectual difficulty. 
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With sufficient care and ingenuity, we could doubtless 
devise composite tasks I, II, III, IV, V, and so on, such that 
there would be percents between 100 and 0 succeeding 
within each of the groups shown above and overlapping of 
groups as shown in the program above. It is probable that 
the number of test composites and of animal groups neces- 
sary will be much less than the program shows. We could 
then measure the differences in difficulty from A down to 
that intellectual task which earth'worms can master, by the 
same general methods as we have used from high levels 
down to A. 

The difficulty of the intellectual task which the earth- 
worm ^s intellect can master is so near zero difficulty that a 
level slightly below it may safely be accepted as an approxi- 
mate absolute zero of intellectual difficulty, sufficiently close 
for all purposes, theoretical or practical. 

It is to be hoped that such a determination of zero intel- 
lectual difficulty by actual experimentation will sometime be 
made. The time and facilities required made it imprac- 
ticable to include it among our investigations. We have 
had to content ourselves with cruder methods. 

LOCATIITG ZERO DIFFICULTY BY A CONSENSUS 

What we have done is to utilize a consensus of psycholo- 
gists, especially such as are expert in animal and infant 
psychology, or the iisychology of the very dull, hlach of 
them in entire independence of all the others ranked 56 
tasks shown below (but presented in a random order) in 
accordance with the following instructions : 

Please rank the tasks or achievements described on the 
enclosed slips according to their intellectual difficulty, that 
is, according to the degree of intellect required for a man to 
perform each, supposing the man to have lived 20 years 
with the average opportunities of a person born and bred 
in an average English-speaking home (or institution, if his 
intellect is so slight that he has to be brought up in an in- 
stitution for the feeble-minded). Mark the task or achieve- 
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ment that requires least intellect, 1; the one that requires 
next to least, 2 ; the one that requires second to least, 3 ; and 
so on. Use your own conception of intellect in doing so. 
If any of the tasks or achievements seem equally difficult, 
assign them the same rank. 


□ 


^ ^ 

Fig. 34. Drawings used with tasks 57, 58, 81, and 82. Beduced to about 

2/3 original size. 

57. Responds correctly to the direction ‘‘Draw a line 
around the cup,*’ Fig. 34a being shown. 

58. Responds correctly to the direction “Make a cross in 
the square,” Fig. 34b being shown. 

69. Can answer correctly “What is the person that you 
send for when you are sick?” 

70. Gan answ'er correctly “Tell me something that walks on 
four legs.” 

75. Can recognize four fingers when four are held up and 
he is asked “How many?” in three trials out of five, the 
trials being interspersed with other tasks. 

24 
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76. The experimenter shows a box is empty and shows five 
pennies clearly and puts them in the box. He then takes 
out four pennies from the box and shows these clearly 
and asks ‘Hiow many cents are in the box?” Two cor- 
rect responses out of three are required, the trials being 
interspersed with other tasks. 

81. Can respond correctly when told “Put your finger on 
the pistol,” or “Point to the pistol,” or “Find the pis- 
tol,” or “Which is the pistol?” or other familiar expres- 
sion of similar meaning, Fig. 34c being shown. 

82. Can respond correctly when told “Put your finger on 
the tiger,” or “Point to the tiger,” or “Find the tiger,” 
or “Which is the tiger?” or other familiar expression 
of similar meaning. Fig. 34d being shown. 

55. Responds correctly to the direction “Make the other 
arm on tliis man.” (Pointing.) 




Fig. 35. Drawing used with task 55. Beduced to about 2/3 original size. 

63. Responds correctly to the direction “Stand back of 
your chair.” 

65. Can answer correctly “Is this morning or afternoon or 
evening? Wliich is it?” 

66. Can answer correctly “Tell me something that you are 
afraid of.” 

74. The experimenter shows that a box is empty. He holds 
up one penny and puts it in the box. He then holds up 
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two pennies, both clearly in view, and puts them in the 
box. He then asks “How many cents are in the box?’* 
(Two correct responses out of three are required, the 
three trials beng interspersed with other tasks.) 

Can respond correctly when told “Put your finger on 
the window, or “Point to the window,’’ or “Find the 
window,” or “Wliich is the window?” or other familiar 
expression of similar meaning. Fig. 36a being shown. 



t’lo. 36, 





Drawings used with i isks 79 and 80. ^Reduced to about 2/3 original 

size. 


80. Can respond correctly when told “Put your finger on 
the envelope,” or “Point to the envelope,” or “Find 
the envelope,” or “^'hich is the envelope?” or other 
familiar exi^ressioii of similar meaning, Fig. 36b being 
shown. 

51. Responds correctly to tli-‘ direction ^‘JMake a line like 
this,” tlie experimenter slowing him by drawing a line 
on a sheet of paper. ^ v.nything approximating a 
straight line is to be scored correct.) 

52. Responds correctly to the direction “Make a cross like 
this.” (Two lines that cross anywhere are to be scored 
correct.) 

59. Can answer “What do you wear on your head when y'ou 

go out?” ; 

60. Can answer “Tell nrfe something that is good to eat. 
Something else. Something else.” (3 required.) 

71. Can respond eorrectl 3 ' to “Show me the littlcst square; 
show me the littlest one of all,” showing three as here. 
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(Three squares with sides respectively i", 1" 
are shown.) 

72. Can respond correctly to Show me the biggest square ; 
show me the biggest one of all,*’ showing three as here. 
(Three squares shown as in 71.) 

77, Can respond correctly when told “Put your finger on 
the apple,” or “Point to the apple,” or “Find the 
apple,” or “Which is the apple?” or other familiar ex- 
pression of similar meaning. Fig. 37a being shown. 



Fio. 37. Drawings used with tasks 77 and 78 Deduced to about 2/3 original 

size. 


78. Can respond correctly when told “Put your finger on 
the dog,” or “Point to the dog,” or “Find the dog,” 
or “WTiich is the dog?” or other familiar expression of 
similar meaning. Fig. 37b being shown. 

45. Can give a correct response to “Tell me the name of 
something you eat.” 

49. Can give a correct response when someone shows him a 
watch and says “Tell me what this is.” 

46. Can give a correct response when someone shows him a 
penny and says “Tell me what this is.” 

4. Responds to the direction “Give me the pencil,” assum- 
ing that a pencil is in clear view before him and that you 
are near enough for him to hand it to you. 

5. Responds to the direction “Take the pencil,” supposing 
one to be in clear viuts within his reach. 
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42i» Can give a correct answer to “What do you call this!*’ 
(the questioner touching the nose of the one ques- 
tioned). 

29. Will not try to put a large object through a hole less 
than one-fourth its size, for example, will not try to put 
a baseball into an inkwell or put a football into his 
pocket. 

50. Can get into bed and cover himself with the bedclothes. 

31. Will be disturbed if, after turning away from two cher- 
ished objects (such as two pieces of cake) he turns 
back to find only one left. 

18. Can open a door by turning an ordinary knob. 

11. Responds to the direction “Shake hands*’ by holding 
out his hand. 

3. Responds to the direction “Hold up your hand.** 

17. Can put on his hat. 

40. Familiar and attractive food being on his plate, he will 
be able to put it in his mouth with a spoon. 

1. Responds to the direction, “Stand up.’* 

7. Responds to the direction, “Come here.” 

33. Can find his way to some very familiar place, such as 
the dining-room, his own bedroom or the bathroom. 

26. Will not walk off a roof or wharf or the like where the 
distance to the ground or water is 20 feet or more. 

10. Responds to the direction, “Come here, John” (sup- 
posing “John” to be his own name and assuming also 
that the person giving the direction is a familiar friend 
speaking in a pleasant voice, with a smile and with open 
arms, representing a very habitual situation to which 
approach has been the response). 

23. Will go toward an object six feet off in case it is a 
familiar, attractive, desired object. 

24. Will go around a familiar object, or push it out of his 
way, if it is movable, that is an obstacle in the way of 
his passage to some attractive, desired, familiar object 
which he is approaching. 
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16. Understands the meaning of **bad hoy/' spoken in a 
harsh voice with disapproving looks. (Use ‘‘girl" for 
“boy" in the ease of a female.) 

39. Familiar and attractive food being on his plate, he will 
be able to get it into his mouth somehow or other. 

35. Being in the presence of some familiar source of great 
heat (wood fire, coal fire, gas flame, or the like, accord- 
ing to his environment), will not put his hand in it. 

28. Will not try to eat a familiar nasty-tasting object, such 
as soap or ashes or a shoe. 

14. Understands his own name so as at least to feel or think 
diiferently when it is spoken than when some other 
word is. 

34. Being in the presence of some familiar, attractive, de- 
sired object, such as food when he is hungry, will go 
toward it rather than away from it. 

41. Will bend his head or body to avoid a blow directed 
from in front straight at his nose. 

32. Responds to his best and kindest friend, for example, 
mother or nurse, differently from his resiwnse to 
strangers. 

27. Will not tr)’^ to pull off his own fingers or toes. 

38. Familiar and attractive food being offered him in a 
familiar 'way, will take it. 

37. Having an object of bitter, nasty taste in his mouth, 
will spit it out more often than hold it there. 

36. Having an object of sweet, pleasant taste in his mouth, 
will keep it there more often than spit it out. . 

A point somewhere between the last two tasks li ^ (36 
and 37) and tasks 27 and 32 may fairly be taken % repre- 
sent approximately zero intellectual difficulty, since tasks 
36 and 37 are comparable to acts done by animsiis that can- 
not learn or can learn only in an infinitesimal Vay, and are 
done by human beings probably without any learning. 
Tasks 27 and 32 (“Will not try to pull off hife own fingers 
or toes" and “Responds to his best and kljndest friend 
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. . .*’) are such as very dull animals can learn. Tasks 51, 
52, 59, 60, 71, 72, 77, and 78 are a- sampling from our level A. 
Tasks 55, 63, 65, 66, 74, 79, and 80 are a sampling from our 
level B. Tasks 57, 58, 69, 70, 75, 76, 81, and 82 are a sam- 
pling from our level C. -We know from experiment the dif- 
ference in difficulty between A and B, and between B and 
C. If we can measure the differences C-B and B-A, we 
have extended our scale to an approximate absolute zero. 
We make this measurement by measuring the following dis- 
tances : 

36, 37 to 27, 32, which are of nearly equal difficulty inter se. 
27, 32 to 14, 28, 35, 39, which are of nearly equal difficulty 
inter se. 

14, 28, 35, 39 to 10, 26, 33, which are of nearly equal diffi- 
culty inter se. 

10, 26, 33 to 3, 11, 18, 31, 50, which are of nearly equal diffi- 
culty inter se. 

3, 11, 18, 31, 50 to 46, 49, which are of nearly equal diffi- 
culty inter se. 

46, 49 to the level A tasks, which are of nearly equal diffi- 
culty inter se. 

The level A to the level B tasks, which are of nearly equal 
difficulty inter se. 

The level B to the level C tasks, which are of nearly equal 
difficulty inter se. 

The measurements, in terms of the size of the minority 
(that is, the number of judges placing the one task as 
harder than the other, though it is in truth easier), are 
shown in Table 125. The corresponding distances in terms 
of the variability of expert opinion, assuming it to be of 
Form A, and taking 1.00 as its Q or Median Variation, are 
also given in Table 125.® 

The difference between 36, 37, and A is 4.7 times the dif- 
ference between A and C. The difference between 27, 32, 

s These values will be in approximately the same proportions by any rea* 
Bonable assumption about the form of the distribution , since we are usin^^ only 
minorities between 6 and 15 out of 40^ 
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and A is 4 times the difference between A and C. We may 
then set the absolute zero of intellectual difficulty provi- 
sionally at A — 4.35 (C — A). C — A equals 5.28ai» by the 
facts reported in Chapter IX, whereby zero is 23019 below 
A or A has a difficulty of 23oi9 measured from zero or just 
not any intellectual difficulty. We estimate the unreliability 
of this 23 due to the small number of judges as a probable 
error of about 2. 


TABLE 125. 

The Estimated Differences in Difficulty of Intellectual Tasks from 
Zero Difficulty to the Difficui/ty of Composite C. 





Average 

Minority 

% 

Difference 
in Diffiealt 7 

36, 37, 

to 

27, 32, 

10.25 

25.6 

.97 

27, 32, 

t€ 

14, 28, 35, 39, 

9.75 

24.4 

1.03 

14, 28, 35, 39, 

f i 

10, 26, 33, 

10.33 

25.7 

.97 

10, 26, 33, 

tt 

3, 11, 18, 31, 50 

9,97 

24.9 

1.00 

3, 11, 18,31, 50 

i i 

46, 49, 

8.40 

21.0 

1.19 

46, 49, 

it 

A 

6.72 

16.8 

1.42 

A 

it 

B6 

10.95 

27.4 

.89 

B 

1 1 

Co 

14.66 

36.6 

.51 

36, 37, 

1 1 

A 



6.58 

27, 32, 

it 

A 



5.61 

A 

it 

O 



1.40 


We present all the facts concerning all the ratings in 
Table 126. The distance from zero to A can be determined 
via different tasks from those which we have used, but the 
result will be substantially the same provided that enough 
tasks are used to reduce the effect of chance. 

Bln the case of the A to R, and B to C comparisons, it would perhaps 
be better to use the median minority rather than the average minority, be- 
cause of the irregularities in the separate comparisons. The two medians 
which would then replace 10.95 and 14.C6 are 9.0 and 15.5. The differences 
in diflSculty would then be 1.02 and .42, giving a total of 1.44 for the O-A 
difference in place of the 1.40. The end result is thus almost the same. If 
the median minority is used for the 46, 49 to A comparison, we have 6.75 in 
place of 6.72. 
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TABLE 126 — (Continued). 
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CHAPTER XI 


The Meastjbbmekt or the Altitude of an Individual. 

Intellect 

It will, of course, very rarely happen that an individual 
will have exactly twenty single tasks right out of the forty 
in any composite. We have to estimate the level at which 
he would have exactly fifty percent right from such a record 
as that shown below. 

Composite Difficulty IJKDMNOPQ 

Numbers Correct 24 18 11 8 1 

We may use all or part of his record, bearing in mind 
that percents near zero and 100 (that is, numbers right out 
of forty near zero and forty) are of less value than those 
nearer 50 percent. To use his record most effectively we 
need to know the general form of the curve (especially near 
the 50 percent point) and to perform the equivalent of plot- 
ting the particular curve of this general form which best 
fits the observations. We have determined the general form 
of the curve from 10 percent of successes to 90 percent of 
successes by the method described below. 

THE FORM OF THE CURVE OF PERCENT CORRECT IN RBI^TION 

TO DIFFICULTY 

All the cases of group 17 and 13 are grouped into five 
groups according to their general altitude as shown by the 
sum of their scores (number right) in N, O, P, and Q. 

The median score at each of the four levels for each of 
the five groups is computed ; and the five curves are drawn. 
They appear in Fig. 38. 

These curves are subject to a very slight error due to 
the fact that they were computed using a value 41.9 for the 
difficulty of composite P, which was later found to be 0.2 off 
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The four curves other than that for the middle group 
are then shifted up or down until each fits the curve for the 
middle group as closely as may be for such part of the range 
from zero to forty correct as they have in common. The 
result is shown in Fig. 39. A curve (Fig. 40) representing 
the central tendency of all the five in Fig. 39 is drawn. Fig. 
40 represents the most probable general form of the curve 
of decrease in percentage correct with increase in difficulty 
so far as groups 17 and 13 reveal it. 



0 5 10 15 20 25 30 35 

Fig« 40. The probable form of the decrease in percent correct with increase 
in diffi-culty, for Grouiis 13 and 17. 

Fig. 40 has heights as follows (in terms of tenths of O 19 ) : 

5 correct, + 30^. 

10 correct, -f- 16 
15 correct, + 7^ 

20 correct, 0 . 

25 correct, — 9. 

30 correct, — 18. 

35 correct, — 31 ^. 
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All the cases of group 91 are grouped into five groups 
according to the sum of their scores in I, J, K, L, and M. 
The median score at each of the five levels is computed for 
each of the five groups; and the five curves are drawn. 
They api)ear in Fig. 41, being curves 6, 7, 8, 9, and 10. 



9-II, 


The four curves for 91 other than that for the middle 
group are then shifted up or down until each fits the curve 
for the middle group as closely as may be for such part of 
the range from zero to forty correct as they have in com- 
mon. The result is shown in Fig. 42. A curve (Fig. 44a) 
representing the central tendency of all the five in Fig. 42 
is drawn. 
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All the cases of group 9II are grouped into five groups 

according to the sum of their scores in K, L, M, N, and O. 

The median score at each of the five levels is computed for 

each of the five groups; and the five curves are drawn. 

They appear in Fig, 41, being curves 1, 2, 3, 4, and 5. 

The four curves for 9II other than that for the middle 

group are then shifted up or down until each fits the curve 

\ 

t 

\ \ 



Fio. 42* The curves for 9-1 (6-10 of Fig. 41) shifted so that similar percents 
correct fall approximately on the same points. 


for the middle group as closely as may be for such i>art of 
the range from zero to forty correct as they have in com- 
mon. The result is shown in Fig. 43. A curve (Fig. 44b) 
representing the central tendency of all the five in Fig. 43 
is drawn. The curve of Fig. 44b is then shifted down so as 
to fit Fig. 44a over that part of the range which they have 
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in common and an average of the two is drawn. This is 
Fig. 45, which represents the most probable general form 
of the curve of decrease in number correct with increase in 
difficulty so far as groups 91 and 9II reveal it. 



\ 

\ 

\ 

i I I i I t I 

0 5 «0 15 zo as so 

Fig. 43. The curves for 9— II (1-5 of Fig. 41) shifted so that similar percents 
correct fall approximately on the same points. 

This curve has heights as follows (in terms of tenths of 

<Fio) • 

5 correct, + 28. 

10 correct, -f- 15. 

15 correct, + 8. 

20 correct, 0. 

25 correct, — 8. 

30 correct, — 18. 

35 correct, — 32. 


25 
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b 

\ 

\ 

\ 


\ 

\ 

\ 



Fig. 44. The central tendency of the five curves of Fig. 42 (a), and the cen- 
tral tendency of the five curves of Fig. 43 (b). 

40 and Fig. 45 are very closely alike in form, and 
each is well represented by a curve with heights (in terms 
of tenths of qio) as follows : 

5 correct, + 30. 

10 correct, -{- 18. 

15 correct, -|- 8. 

20 correct, 0. 

25 correct, — 8. 

30 correct, — 18. 

35 correct, — 30. 

All the cases of Groups 4 and 5 are grouped into six 
groups according to the sum of their scores in H, I, J, and 
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K. The six curves are drawn (Fig. 46), and shifted up or 
down so that similar numbers correct fall on approximately 



correct fall approximately on the same points. 


the same points (Fig. 47), and combined to form a curve 
(Fig. 48) representing the central tendency for number cor- 
rect in relation to increase in difficulty in the same way that 
Fig. 40 was formed from the facts of Fig. 38. This curve 
has heights as follows (in terms of tenths of Oio) : 

5 correct, + 28J. 

10 correct, -f* 154- 

15 correct, -f- Of. 

20 correct, 0. 

25 correct, — 7. 

30 correct, — 14. 

35 correct, — 244. 
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The cases of Group im3 are grouped into five groups 
according to the sum of their scores in A, B, C, and D ; and 
the same procedure followed as hitherto. The facts appear 

L — 



Fig. 46. CuTves of percent correct in relation to difficulty; Groups 4 and 5. 

in Figs. 49, 50, and 51. Fig. 51 has heights as follows (in 
terms of tenths of Ois) ^ 

5 correct, + 34^. 

10 correct, +21. 

15 correct, + 9|. 

20 correct, 0. 

25 correct, — 9^. 

30 correct, — 22. 

35 correct, — 36. 

The cases of Group im6 are grouped into three groups 
according to the sum of their scores in C, D, E, F, and G; 
and the same procedure followed. The facts appear in 



TBE ALTITUDS OF AJST INDIVlDUAIi ZNTEIiLEOT 363 

Mgs. 52, 53, and 54. Fig. 54 has heights as follows (in 
terms of tenths of Oi») : 

5 correct, + 16. 

10 correct, +11. 

15 correct, + 6. 

20 correct, 0. 

25 correct, — 6^. 

30 correct, — 15^. 

35 correct, — 31^ (approx.) 

The facts whence Figs. 38 to 54 are derived are pre- 
sented in Table 127. 



Fig. 47. The euTves of Fig. 46 shifted so that similar percents correct fall 

on approximately the same points. 
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From Composite Q down to Composite I, or from the 
College-Graduate group down through Grade 5, there is a 
notable similarity not only in the shape but also in the 
amount of slope of the curves. From 5 correct to 35 cor- 
rect equals 6.2ai9 at or near Composites O and P, 6.0oifl at 
or near Composites L and M, and 5.3<Tio at or near Com- 
posites J and K. At lower levels there is considerable 



48. The probable form of the decrease in percent correct with increase 
in difficulty, for Groups 4 and 5. 

disparity, the spread around Composites B and C being 
T.Ooi®, while that around Composites E, F, and G is only 
about 4.80,9. The determinations around levels A to G 
are much less reliable than those from I to Q. A combina- 
tion of Figs. 51 and 54 with equal weight gives (in terms 
of tenths of 0,9) -f 25, -f 16, + 8, 0, — - 8, — 19, and — 34, 
with a spread of 5.90,9. 
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ESTIMATING THE OAVD ALTITUDE OP AN INDIVIDUAL 

From I to Q the altitude at which an individual will 
have exactly twenty right out of forty can be estimated 
from Table 128, which approximately represents a curve 
of the general form of Figs. 40, 45, and 48, running from 
+ 3.0(718 at 5 correct to — S.Oois at 35 correct. 



Pia. 49. Curves of percent correct in relation to difficulty; Group im 3. 


Table 129 is a similar table for Composites A to H, 
which may be used provisionally until the difficulties of 
these tasks and the nature of the relation between difficulty 
and number correct at these levels are more accurately de- 
termined. 






THE ALTITUDE OF AN" IISTDIVIDUAL INTELLECT 


369 


Any 40-composite task at which an individual has more 
than zero and less than forty of the single tasks correct 
can be used to estimate the level or altitude at which he 
would probably have exactly 20 right. Scores near zero or 
forty deserve, however, r^atively little weight, because the 
former are so much affected by chance successes in the case 
of the vocabulary tasks, and the latter are so much affected 
by careless slips. 

A simple and impartial procedure which works very well 
in general is to utilize for any individual the three succes- 



Fia. 54. The probable form of the decrease in percent correct with increase 

in difficulty, for Qroup im 6. 

sive levels whose sum of rights is nearest to 60, and to attach 
equal weight to each of them. This is the procedure used 
in all the measurements of altitude which are reported in 
later chapters. 

A possibly better procedure is to take the two successive 
levels whose sum of rights is nearest 40, and the neighbor 
ing level above or below which is nearest 20. This seems to 
be somewhat quicker in operation, and is less likely to in- 
clude levels with extremely low or high percents correct. 



TABLE 127. 

Mkdiak or THs ScosES (NniiBKE Bight otit or 40) ros Each or Twentt-Ninx Gbotjts with Each or Foub ob More Neigh- 
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13 f 17a 16.3 9.7 6.5 4.0 

b 24.8 17.2 12.5 6.0 

e 28.7 22.6 18.4 10.5 

d 32.0 27.7 24.0 16.5 

• 35.1 31.3 29.7 21.3 
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TABLE 128. 

Altititdes Corresponpino to Any Number Ck>RREGT from 5 to 35 Our of 40 

FOR Tasks I to Q. 



I 

j 

K 

L 

M 

N 

o 

P 

Q 

5 

31.2 

33.1 

34.2 

35.2 

36.2 

36.9 

38.1 

38.7 

40.1 

6 

31.4 

33.3 

34,4 

35,4 

36.4 

37.1 

38.3 

38.9 

40.3 

7 

31.7 

33.6 

34,7 

35,7 

36.7 

37.4 

38.6 

39.2 

40.6 

8 

31.9 

33.8 

34.9 

35.9 

36.9 

37.6 

38.8 

39.4 

40.8 

9 

32.2 

34.1 

35.2 

36.2 

37.2 

37.9 

39.1 

39.7 

41.1 

10 

32.4 

34.3 

35.4 

36.4 

37.4 

38.1 

39.3 

39.9 

41.3 

11 

32.6 

34.5 

35.6 

36.6 

37.6 

38.3 

39.5 

40.1 

41.5 

12 

32.8 

34.7 

35.8 

36.8 

37.8 

38.5 

39.7 

40.3 

41.7 

13 

33.0 

34.9 

36.0 

37.0 

38.0 

38,7 

39.9 

40.5 

41.9 

14 

33.2 

35.1 

36.2 

37.2 

38.2 

38,9 

40.1 

40,7 

42.1 

15 

33.4 

35.3 

36.4 

37,4 

38.4 

39.1 

40.3 

40.9 

42.3 

16 

33.6 

35.5 

36.6 

37-6 

38.6 

39.3 

40.5 

41.1 

42.5 

17 

33.7 

35.6 

36.7 

37.7 

38.7 

39.4 

40.6 

41.2 

42.6 

18 

33.9 

35.8 

36,9 

37.9 

38.9 

39.6 

40.8 

41.4 

42.8 

19 

34.0 

35.9 

37.0 

38.0 

39.0 

39.7 

40.9 

41.5 

42.9 

20 

34.2 

36.1 

37.2 

38.2 

39.2 

39.9 

41.1 

41.7 

43.1 

21 

34.4 

36.3 

37.4 

38.4 

39.4 

40.1 

41.3 

41.9 

43.3 

22 

34.5 

36.4 

37.5 

38.5 

39.5 

40.2 

41.4 

42.0 

43.4 

23 

34.7 

36.6 

37.7 

38.7 

39.7 

40.4 

41-6 

42.2 

43.6 

24 

34.8 

36.7 

37.8 

38.8 

39.8 

40.5 

41.7 

42.3 

43.7 

25 

35.0 

36.9 

.38.0 

39.0 

40.0 

40.7 

41.9 

42.5 

43.9 

26 

35.2 

37.1 

38.2 

39.2 

40.2 

40.9 

42.1 

42.7 

44.1 

27 

35.4 

37.3 

38.4 

39.4 

40.4 

41.1 

42.3 

42.9 

44.3 

28 

35.6 

37.3 

3S.6 

39.6 

40.6 

41.3 

42.5 

43.1 

44.5 

29 

35.8 

37.7 

38.8 

39.8 

40.8 

41.5 

42.7 

43.3 

44.7 

30 

36.0 

37.9 

39.0 

40.0 

41.0 

41.7 

42.9 

43.5 

44.9 

31 

36.2 

38.1 

39.2 

40.2 

41.2 

41.9 

43.1 

43.7 

45.1 

32 

36.5 

38.4 

39.5 

40.5 

41.5 

42.2 

43.4 

44.0 

45.4 

33 

36.7 

38.6 

39.7 

40.7 

41.7 

42.4 

43.6 

44.2 

45.6 

-34 

37.0 

38.9 

40.0 

41.0 

42.0 

42.7 

43.9 

44.5 

45.9 

35 

37.2 

39.1 

40.2 

41.2 

42.2 

42.9 

44.1 

44.7 

46.1 


In the rare cases where it is desirable to estimate altitude from a score 
below 5 or alx>ve 35, the foUowing* may be used: For 1, 2, 3 and 4, subtract 
4.3, 3-9, 3.6 and 3.3 respectively from the score for 20. For 36, 37, 38 and 39, 
add 3.3, 3.6, 3.9 and 4.3 respectively to the score for 20. 
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TABLE 129. 

Appeoximate Provisional Altitudes Corresponding to Ant Number Correct 


FROM 5 TO 35 Out op 40 por Tasks A to n . 


A 

B 

c 

13 

E 

P 

G 

n 

5 

20,0 

23.2 

25.3 

26.7 

27.8 

28.3 

29.0 

29.8 

6 

20.2 

23.4 

25.5 

26.9 

28.0 

28.5 

29.2 

30.0 

7 

20.5 

23.7 

25.8 

27.2 

28.3 

28.8 

29.5 

30.3 

8 

20.7 

23.9 

26.0 

27.4 

28.5 

29.0 

29.7 

30.5 

9 

21.0 

24.2 

26.3 

27.7 

28.8 

29.3 

30.0 

30.8 

10 

21.2 

24.4 

26.5 

27.9 

29.0 

29.5 

30.2 

31.0 

11 

21.4 

24.6 

26.7 

28.1 

29.2 

29.7 

30.4 

31.2 

12 

21.6 

24.8 

26.9 

28.3 

29.4 

29.9 

30.6 

31.4 

13 

21.8 

25.0 

27.1 

28.5 

29.6 

30.1 

30.8 

31.6 

14 

22.0 

25.2 

27.3 

28.7 

29.8 

30.3 

31.0 

31.8 

15 

22.2 

25.4 

27.5 

28.9 

30.0 

30.5 

31.2 

32.0 

16 

22.4 

25.6 

27.7 

29.1 

30.2 

30.7 

31.4 

32.2 

17 

22.5 

25.7 

27.8 

29.2 

30.3 

30.8 

31.5 

32.3 

18 

22.7 

25.9 

28.0 

29.4 

30.5 

31.0 

31.7 

32.5 

19 

22.8 

26.0 

28.1 

29.5 

30.6 

31.1 

31.8 

32.6 

20 

23.0 

26,2 

28.3 

29.7 

30.8 

31.3 

32.0 

32.8 

21 

23.2 

26.4 

28.5 

29,9 

31.0 

31.5 

32.2 

33.0 

22 

23.3 

26.5 

28.6 

30.0 

31.1 

31.6 

32.3 

33.1 

23 

23.5 

26.7 

28.8 

30.2 

31.3 

31.8 

32.5 

33.3 

24 

23.6 

26.8 

28,9 

30.3 

31.4 

31.9 

32 .C 

33.4 

25 

23.8 

27.0 

29.1 

30.5 

31.6 

32,1 

32.8 

33.6 

26 

24.0 

27.2 

29.3 

30.7 

31.8 

32.3 

33.0 

33.8 

27 

24.2 

27.4 

29.5 

30.9 

32.0 

32.5 

33.2 

34.0 

28 

24.4 

27.6 

29.7 

31.1 

32.2 

32.7 

33.4 

34.2 

29 

24.6 

27.8 

29.9 

31.3 

32.4 

32.9 

33.6 

34.4 

30 

24.8 

28.0 

30.1 

31.5 

32.6 

33.1 

33.8 

34.6 

31 

25.0 

28.2 

30.3 

33.7 

32.8 

3.3.3 

34.0 

34.8 

32 

25.3 

28.5 

30.6 

32.0 

33.1 

33.6 

34.3 

35.1 

33 

25.5 

28.7 

30.8 

32.2 

33.3 

33.8 

34.5 

35.3 

34 

25.8 

29.0 

31.1 

32.5 

33.6 

34.1 

34.8 

35.6 

35 

26.0 

29.2 

31.3 

32.7 

33.8 

34.3 

35.0 

35.8 



CPIAPTER XII 


The Measurement of Width and Area of InteijLect 
The width or range of intellect at any altitude or level 
of difficulty is measured by the number of tasks mastered 
at that altitude. Thus, suppose that Intellect X is mea- 
sured with ten 40 -composite tasks (Ni, Ng, Ng, etc.), each 
equal to Composite N in difficulty; and has the following 
score : 


Number of single tasks 

right 

in 

N, 

= 20. 

it it it 

i t 

it 

N^ 

= 19 . 

it it ii it 

ii 

i t 

N« 

= 21. 

it if it if 

ft 

i i 

N4 

= 20. 

a if ft if 

i f 

it 

N5 

= 20. 

it it a it 

if 

it 

Ne 

= 18 . 

ti a it if 

i i 

i i 

N, 

= 19 . 

it if if if 

if 

i i 

Nh 

= 21. 

it if ft ti 

i t 

i i 

No 

= 22. 

it if it ft 

it 

i i 

Nio 

= 20. 


Success at one of these 40 -composite tasks means attain- 
ing 20 or more single tasks correct. The width of Intellect 
X at Altitude N is 7 out of 10 for Tasks Ni to Nio. It may 
also under certain conditions be considered as 200 out of 
400 for the single tasks composing Ni to Ni®, or as a certain 
number out of 40 for the same single tasks grouped in 10 - 
composites, or as a certain number out of 100 for the same 
tasks grouped in 4 -composites. 

WIDTH or intellect in the case of truly intellectual 

tasks 

Consider first the first and most correct meaning, that 
is, the number of composite-tasks correct, here 7 out of 10 
for Tasks Ni to N^o. If the ten are a representative sam- 
pling of tasks of intellectual difficulty N, Intellect X may 
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be expected to have approximately 70 successes out of 100, 
or 700 out of 1,000, or in general approximately 70 percent 
of successes with tasks at the intellectual altitude of N. If 
there are 200 such tasks, his probable width is 140; if there 
are 60,000 such, his probable width is 42,000. If, when mea- 
sured in respect of ten 40-composite tasks representative 
of intellectual difficulty M, his scores are 25, 25, 22, 24, 26, 
25, 23, 24, 26, and 21, he may be expected to have 100 per- 
cent of successes with tasks of intellectual difficulty M. If 
there are 150 such tasks, his probable Avidth is 150. If there 
are 40,000, his probable width is 40,000. 

This illustration directs our attention to two meanings 
of width, namely, width of intellect in the sample examined 
and width of intellect in the entire series which the sample 
represents; and also to the fact that the sample examined 
may have a larger representation of tasks at one altitude 
than of tasks at another. 

Suppose, for example, that the sample contains 

40 single tasks between difficulty 30.0 and 30.99, 

40 35.0 and 35.99, and 

40 ‘‘ ‘‘ ‘‘ 40.0 and 40.99, and 

that there really are one million CAVD tasks between 30.0 
and 30.99, two million between 35.0 and 35.99, and three mil- 
lion between 40.0 and 40.99. Then the sample has twice as 
large a representation of level 35.0 to 35.99 as it has of level 
40.0 to 40.99, and three times as large a rejircsentation of 
level 30.0 to 30.99 as it has of level 40.0 to 40.99. If an indi- 
vidual can do 9 out of 10 of the sample at level 30.0 to 30.99, 
he can probably do 500,000 tasks at that level. But if he 
can do half of the tasks of the sample at level 40.0 to 40.99, 
he can probably do 1,500,000 tasks at that level. 

If each of the tasks, the number of which measures 
width, is perfectly intellectual, depending for success upon 
all of intellect and nothing but intellect, the change from 
one hundred percent of successes to zero percent of suc- 
cesses, as the intellect in question is tested at higher and 
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higher altitudes, will be instantaneous. When a small 
amount of inadequacy and error is present, as in our 40- 
composites for Intellect CAVD, the change will still be very 
sudden. The conditions in representative intellects, each 
measured by a score or more of tasks like our 40-composites 



Fio. 55. The probable perceiitagea of successes of three intellects, I, II 
and III, in a series of 360 tasks, 20 of difficulty A, 20 of difficulty B, and 
BO on, each task having = approximately .9. The drawings are not from 
precise computations, being for illustration only, not for mensuration of the 
effect. 

26 
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at each altitude in Intellect CAVD, will be roughly as shown 
in Fig. 55. 

The evidence for this is the correlations between one 40- 
composite and another at or near the same level, and the in- 
frequency of reversals from failure to success in our series 
of tasks. For example, in the 240 individuals of Group 17, 
of those failing with P (103 in all), only 4 or 3.9 percent 
succeeded with Q, which is 1.1 harder. 

In the 246 individuals of Group 91, of those failing with 
K (93 in all), only 9 or 9.7 percent succeeded with L, which 
is 1.0 harder. 

The measurement of Cx\.VD width at any altitude, in the 
rigorous sense of number of intellectual tasks mastered at 
that altitude, is thus given for most altitudes by the mea- 
surement of altitude itself. Nearly up to that altitude the 
percent is one hundred; above it the percent very soon 
drops to zero. Within the short distance of uncertainty 
the widths may be determined by experiment or estimated 
fairly closely from the altitude. 

This will hold true of any sort of intellect defined and 
treated in the same manner as Intellect CAVD. In propor- 
tion as each task depends for success upon all of intellect 
and nothing but intellect, a smaller and smaller increase in 
difficulty will cause a shift from success to failure, the alti- 
tude where it does so varying with the intellect that is being 
measured. 

WIDTH OF INTELLECT IN THE SENSE OF THE NUMBER OF SINGLE 

SHORT TASKS MASTERED, ANY ONE OF THESE TASKS BEING 
ONLY A VERY PARTIAL REPRESENTATION OF INTELLECT 

For many purjioses it is desirable to know how many 
single tasks from a set which are nearly or quite alike in 
difficulty and which are nearly or quite as intellectual as 
any short single tasks can be, a given intellect can succeed 
with. If, for example, two intellects A and B have identical 
CAVD altitudes exactly at Level N, and if A has average 
scores at Levels K, L, M, N, O, P, and Q of 39, 36, 29, 20, 17, 
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11, and 6, whereas B has scores of 30, 28, 27, 20, 18, 6, and 0, 
there is a difference between A and B which may need ex- 
pression. Between 40 and 20 right, and between 19 and 
zero right in the case of such 40-composites as the CAVB 
series, there are ranges of difference which may be of great 
importance for theory or for practice or for both. 

The measurement is, of course, a simple count of suc- 
cesses in the sample used in the examination, and an esti- 
mated count for the entire series which is represented by 
the sample- If the single tasks in K represent a selection 
of 40 out of 10,000, while those in L represent a selection of 
40 out of 15,000, and those in M represent a selection of 40 
out of 25,000, A’s scores of 39, 36, and 29 in the examina- 

39 36 

tion mean probabilities of success with X 10,000, ^qX 
29 

15,000, and X 25,000, or with 9,750, 13,500, and 18,125 

single tasks of the sort chosen as components of Com- 
posites K, L, and M, respectively. 

A series of names is needed to designate different sorts 
of width, from the width of an intellect in perfectly intel- 
lectual tasks, down through its width in various composite 
tasks less and less representative of all of intellect and 
nothing but intellect, to its width in such tasks as giving the 
opposite of one word, or understanding one sentence, or 
tracing a way through one maze, or repeating one series of 
five digits backward. We suggest the use of a series of 
W’s, each followed by a notation describing the tasks, and 
being in each case the percent of successes. 

Thus, W(10C -h lOA -f- lOV + lOD) would refer to the 
percent of successes with 40-composite tasks made up 
equally of C, A, V, and D; W(1C or lA or IV or 1D)N 
would refer to the percent of successes with a series of tasks 
made up of single C’s, A’s, V’s, and D’s. W(IOM) would 
refer to the percent of successes with a series of composite 
tasks each made up of ten mazes. The altitude at which W 
is measured will require very careful description in every 
case. 
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AREA OF INTELLECT 

Area or volume seems the best term to use to mean the 
total number of tasks of some specified sort at which an in- 
tellect succeeds; and area seems preferable. Area, like 
width, will have two distinct meanings, namely, the number 
of successes in the sample set of tasks examined, and the 
number of estimated successes in the entire inventory of 
tasks which have been or can be made, and of which the ex- 
amination-tasks are a representative sample. 

Area of intellect, like width, is, in the strictest usage, 
the number of truly intellectual tasks, each of wdiich mea- 
sures all of intellect and nothing but intellect. In this sense 
the area found will be a function of the altitude; Intellect 
X, of Altitude N, will succeed with all tasks uj) to that alti- 
tude, and with none beyond it. 

As in the case of width, it will be desirable to use area of 
intellect in a loose sense to mean the total number of tasks 
mastered which are iiroper components of composites 
which, as totals, are intellectual, all the way down from 
composites which arc nearly perfectly intellectual to short 
single tasks like the single C’s, A’s, V’s, and U’s. A nota- 
tion like A(10C-l-10A-f lOV + lOD), A (1C or lA or IV 
or 1D)N, A(IOM), and the like may usefully be adopted to 
describe the kind of “area” that is being measured. 

We shall consder as a typical case the measurement of 
A(1C or lA or IV or ID). Everything is simple so far as 
concerns finding this area for the sample examined. But 
the effort to estimate the area as a fraction of all the dif- 
ferent sentence-comyjletions that might be desired, all the 
different arithmetical problems which could be collected or 
invented, all the word-knowledge tasks (Shall other than 
English words be used?) jiossible, and all the sentences or 
paragraphs or books that might be heard or read, and so to 
estimate effective A (1C or lA or IV or ID) brings us up 
squarely against great difficulties due to lack of knowledge 
of the relative frequency of different C’s, A’s, V^s, and D’s 
at different levels of difficulty. 
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If we know the width of an intellect at each level in an 
adequate sample of tasks, we can measure its total ^*area,’' 
provided we know the number of tasks at each level. Thus, 
if the C, A, V, and D single tasks of Intellect CAVD at 
levels zero to forty^ number, in order, 100, 100, 100, 100, 100, 
200, 200, 200, 200, 200, 300, 300, 300, 300, 300, 400, 400, 400, 
400, 400, 500, 500, 500, 500, 500, 700, 700, 700, 700, 700, 1000, 
1000, 1000, 1000, 1000, 2000, 2000,2000, 2,000, and 2000, and 
if Intellect JS, when measured with a representative sam- 
pling of 40 at each level, scores 40 at each level up through 
level 30, and 38, 32, 24, 20, 10, 4, 0, 0, 0, 0 in order there- 
after, we find his A (1C or lA or IV or IT)) as 14,200 out of 
a possible 26,000. If there had been 650 tasks at each level, 
the same record in the examination would have meant 
21,580 out of a possible 26,000. 

Such a computation of the area of an intellect would not 
be a mere theoretical curiosity or statistical tour dc force, 
but would be a systematic and accurate way of measuring 
something of very great importance. Common-sense 
thought and action about intellect often deal with some- 
thing which this concept of area makes definite and objec- 
tive. Just as terms like acuity, originality, and intellectual 
genius refer to intellect with especial emphasis on its alti- 
tude, so terms like breadth, scope, and intellectual pow^er 
refer to intellect with especial emphasis on its “area.” We 
should not expect common sense to make clean-cut distinc- 
tions or to avoid confusions, for the very good reason that 
altitude and area are closely correlated, so that for most 
practical purposes, we can describe a man’s intelligence 
adequately by simply rating him for intelligence as a unit. 
But the concept of a man’s general average probability of 
correct response to intellectual or semi-intellectual tasks 
has been real and useful ; and it will be more so now that it 
can be made definite and measurable. 

1 Lievel 0 includes all C, A, V or D tasks from 0 difficulty up to a difficulty 
of 1.00^ 1 includes all from I.OO up to 2.00^ 2 includes all from 2.00 up to 3.00, 
and BO on* 
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It is possible to discover approximately the number of 
single tasks at each level of Intellect CAVD or any other 
defined intellect, though such estimates are beset by many 
difficulties. The enumeration of the C or A or V or D tasks 
harder than the average of those in Composite N and easier 
than the average of those in Composite O is, indeed, prob- 
ably comparable in complexity to the enumeration of all 
the species of animals. 

The chief and most obvious difficulty is that of deciding 
how much one task must differ from another in order that 
they shall be counted as two rather than one. Consider, for 
example, these fourteen tasks to be given orally: 

1. John is 5 years old now. How old will he be in 3 years t 

2. Tom is 5 years old now. How old will he be in 3 years'? 

3. John is 5 years old now. Tom is 3 years older than 

John. How old is Tom? 

4. John is 5 years old. WiU is 3 years older than John. 

How old is Will? 

5. John has 5 cents now. How much will he have if his 

father gives him 3 cents ? 

6. John has 5 cents now. How much will he have if his 

mother gives him 3 cents? 

7. How many dollars are five dollars and 3 dollars? 

la. John is 6 years old now. How old will he be in 3 years ? 
2a. Tom is 6 years old now. How old will he be in 3 years? 
3a. John is 6 years old now. Tom is 3 years older than 
John. How old is Tom ? 

4a. John is 6 years old. Will is 3 years older than John. 
How old is Will? 

5a. John has 6 cents now. How much vill he have if his 
father gives him 3 cents? 

6a. John has 6 cents now. How much will he have if his 
mother gives him 3 cents ? 

7a. How many dollars arc 6 dollars and 3 dollars? 

How many different tasks are there? All competent 
students of intellect will deny that there are fourteen. 
By any reasonable view, we should not count 2 as a dif- 
ferent arithmetical task from 1. Whether the problem is 
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put about John or Tom or Will or Mary, does not, we 
think, make any difference to it as an arithmetical or intel- 
lectual task. Our thinking is probably sound, and we shall 
later state the facts and principle which justify it. But 
note that if we think in a stiff pseudo-logical way that the 
name of the boy makes no difference, we shall err. Let 
Tom, well known to be of age ten, be sitting in full sight 
and the task is now not quite the same, requiring for suc- 
cess that the intellect shall not be misled by the temptation 
to think of the present Tom. Or let the problem be stated 
as “Sneezer Snoop Squibb is 5 years old now. How old 
will Sneezer Snoop Squibb be in 3 years?*' and the task is 
not quite the same, requiring that the intellect be not dis- 
tracted by the seductive name into inattention to the num- 
bers. 

If a psychologist should list all the arithmetical tasks 
that ever have been set, and add to them all that a decade 
of ingenious thought could devise, and then try to cull out 
the duplicates, he would find some that would be indubitably 
so, and some that would be as unlike as arithmetical tasks 
can be ; but with many he could only say that the two tasks 
were somewhat different. So he would have to set up some 
standard of the amount of difference which would qualify 
two tasks to be counted as two, or some scheme for frac- 
tional counts. 

The facts which he should use for these purposes are the 
facts of the differences of the tasks as tasks for intellects. 
By this is meant not merely that the facts are facts in the 
minds or neurones of individuals, but also that they are 
facts in the action of intellects to which the tasks are pre- 
sented for solution. Two sentences differing in print only 
by an apostrophe or comma may differ enormously in the 
intellectual actions which they evoke in an intellect set upon 
solving them, and two questions which have not a word in 
common may arouse very similar behavior, as is the case 
with “Solve 2x*-}-x = 21’' and “What does y equal if 
2y* + y = 21?” for competent students of algebra. And 
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either may arouse very different behavior according as the 
person reacts to it as a task to be solved or as, say, a mere 
question to contemplate. 

So the investigator seeking to measure the differences 
amongst tasks (apart from differences in difficulty) must 
be expert in the psychology of thinking, and must be skilful 
in examining and cross-examining individuals who do the 
tasks in question and report what they did. He will often 
have to make subtle distinctions in cases where two tasks 
arouse different action in two intellects, and when it is 
doubtful how much of the difference lies in the tasks and 
how much in the intellects. 

The objective method of correlation will be helpfuL Ob- 
viously, if an intellect can do task 1 and cannot do task 2, 
then the two tasks are different for that intellect ; two tasks 
are not perfectly alike as tasks for intellect, unless every 
intellect that can do one can do the other. Other things 
being equal, the more individuals there are within any given 
group who can do the one task and not the other, the greater 
the difference between the two (for that group) will be. 
More generally, the differences with which we are concerned 
here are measured, other things being equal, by the lack of 
I)erfect correlation between the ability to succeed with one 
and the ability to succeed with the other, in some defined 
group of intellects, difficulty being kept constant. If two 
tasks are identical as tasks for intellects, rtna will be 1.00. 
If they are of equal difficulty, the more unlike they are the 
lower rtit 2 will be until it reaches a minimum which repre- 
sents the amount of likeness which two tasks must have to 
be included in the series of arithmetical tasks which is to 
be enumerated. 

This argument from correlation will not hold good if a 
task is a composite where success is defined as obtaining a 
certain percent of successes with the elementary tasks, or 
attaining a certain score by some system of credits. Two 
such composites may show very high correlations in respect 
of success as just defined, and yet have hardly a single 
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detail of one like any detail of the other. The correlations 
are between scores, each of which measures chiefly ability 
in what is summed up in or common to all the single tasks 
of the composite, not what is characteristic of any one of 
them as a totality. The composites are closely alike in the 
sense that what is summed up in or common to all the single 
tasks of A, is closely like what is summed up in or common 
to all the single tasks of B. 

PROPORTIONAL COUNTS 

For some purposes, the relative numbers of tasks at the 
different levels of difficulty will serve in place of the abso- 
lute numbers. Thus, if we wish to know what percent of 
A’s area B*s area is, we will do as well by knowing that the 
numbers of tasks are in the proportions Iv, 3K, 9K, 27K, 
81K, as by knowing their absolute amounts. 

Jt may well be that sueli proportional counts may be 
made with greater accuracy, as well as with greater ease 
and speed, than absolute counts. Certain factors of error 
may act alike at all levels and so do no harm to the propor- 
tional counts. Certain arbitrary schemes of fractional al- 
lowance for overlapping tasks may also act alike at all 
levels and so do no harm to proportional counts. Kven such 
proportional counts, however, will require much sagacity 
and industry to acliieve even approximate truth for even a 
small fraction of intellect. A reasonably satisfactory pro- 
portional count of the number of tasks at each level of even 
so small a representation of intellect as CAVD is, will in- 
deed require an enormous amount of observation and ex- 
periment. New tasks, like new species of animals, are 
coming into existence while we count them; tasks a and c 
seem enough different to count as two, and tasks b and d 
seem enough different to count as two, but when a, b, c, and 
d are considered together, they do not seem to deserve a 
credit of four; it seems as if some sorts of tasks at some 
levels of difficulty were innumerable ; when task X is simply 
a task where both a and b must be performed successfully 
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to bring success in X, shall we count a and b and X as 3 or 
as 2? These difficulties, together with those which have 
been mentioned and others which might be, may make per- 
fect or even approximately correct counts impossible. 

The best way to find out what is possible is to begin 
work at actual counts. We have begun, but have not 
progressed far enough to report results, save one. This 
one, which the reader ^s sagacity may have anticipated, is 
that the number of different tasks per unit of altitude of 
intellect is not equal, but increases as we go up from zero 
altitude. 

That this is true for sentence completions can be easily 
realized if one will try to make as many different C tasks 
as he can between the average difficulty of those in A and 
the average difficulty of those in B (23 to 26 ^), and to do 
the same for the stretch of difficulty from N to Q (40 to 43). 
It will be found very hard and perhaps impossible to devise 
five hundred of the former, whereas there seems almost no 
limit to the possible number of the latter. Apparently the 
harder the task, the greater the number possible, though it 
is not easy to devise extremely hard completions which are 
linguistic rather than informational in their difficulty. 

In the case of the arithmetical tasks the number of dif- 
ferent tasks surely increases from the very easy levels up 
to a certain point, after which there is some doubt. The 
doubt seems, however, to be due mainly to our averseness 
to fabricating problems whch are so elaborate and intricate 
as to be extremely unreal, rather than to the paucity of 
such. In the case of the disarrangcd-equation task, it is 
obvious that the number of different ones possible increases 
rapidly with increased difficulty and has no limit. 

In the ease of vocabulary, the fact is unquestioned if 
other languages than English are included, and probably 
holds true for English alone. 

In the case of the understanding of sentences and para- 
graphs, the increase is obscured by the facts that people 
usually try to make their statements as easily intelligible 
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as may be, and that the number of persons who are con- 
cerned with very subtle and intricate ideas is few. Also, 
the number of different statements and questions of even a 
moderate degree of difficulty which can be fabricated is so 
enormous that comparisons are very difficult. Also, only 
persons of very high directions ability can frame state- 
ments which are sensible and correct but very hard to un- 
derstand, and still free from any great informational dif- 
ficulty. Sentence comprehension cannot, however, well be 
kept distinct from informational abilities ; and if informa- 
tional difficulties are allowed to enter freely, the number of 
sentences very hard to understand is practically infinite. 
Even if one abstains from these rather rigorously, the num- 
ber of very hard D’s that can be made is enormously 
greater than one would expect from the number found in 
reading. Merely by combining and permuting causal, con- 
ditional, and concessive clauses and pronoun references, 
one can produce an enormous number of different tasks 
like, * ‘ A change in ab would cause a similar change in og if 
ek did not produce its usual effect upon il, although ek did 
act upon um, and ba would cause an increase in ab, provided 
bi did not occur in unison with bo. What will happen to og 
if ba and bi and ek happen shortly subsequent to bo, pro- 
vided the ek-il action is neutralized by bo, and um does not 
occur?” 

We have not even begun a count for the entire series of 
tasks which might reasonably be made constituents in com- 
posites designed to measure intellect in general. Conse- 
quently, we are not able to make more than a very rough 
estimate of how much number increases with altitude, or of 
the way in which the increase comes. We think the increase 
for Intellect CAVD is so great as to make the number of 
different tasks at level 40 to 40.99 at least a hundred times 
the number at level 20 to 20.99. We also think that it comes 
smoothly and with acceleration, at least up to a certain 
level, after the pattern of Fig. 56 or Fig. 57 or Fig. 58. In- 
tellect CAVD can hardly be said to have an appreciable 
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area below level 20, since it probably requires an altitude 
of 20 to complete ten out of any twenty sentences, no matter 
how easy, or to solve ten out of any twenty arithmetical 
tasks. The increase for intellect in general will be found, 
we think, to increase to a similar degree and in a similar 
manner, with at least fifty times as many tasks at 40 as at 
20, and at least several hundred times as many at 40 as at 10. 

An intellect of altitude 40 may then have an area, not 
twice that of an intellect of altitude 20, but ten or twenty or 
perhaps two hundred times it. The common-sense view 
that the greatest intellect of a thousand men is many times 
as great as the worst intellect of the thousand may be en- 
tirely correct, if we mean bj’^ great'* something corre- 
sponding to area. 

Moreover, if we think of intellect as a hierarchy of unit 
connections or bonds between ideas or between the neural 
correspondents of ideas, the number of different connec- 
tions required to enable a person to respond correctly to 20 
out of 40 of the elements of task N at level 40 may be not 
twice the number required to enable one to respond cor- 
rectly to 20 out of 40 of a task 3 below A, but ten or twenty 
or two hundred times it. 

Intellectual altitude, by our definition, shows a small 
relative rise from the imbecile to the average and then to 
the gifted adult, by the argument followed in Chapter IX, 
so small as to arouse astonishment and incredulity concern- 
ing the usefulness of the definition and the validity of the 
argument, at first thought. If, however, the altitudes of the 
imbecile, average and gifted were in the proportions of 5, 
15, and 20, or 1, 11, and 16, instead of about 25, 35, and 40, 
we might find the relative areas of intellect in the three 
groups much more preposterous in the reverse way. The 
scale of altitude must not be criticized for the lack of at- 
tributes which are appropriate only for a scale of area, 
unless it can be shown that width is approximately the 
same at all altitudes. It is not. 



CHAPTER XIII 


The Relations of Altitude to Width, Area, and Speed 
The miniber of CAVD tasks at any given level of diffi- 
culty is unknown. Consequently all the relations with 
width which are considered in this chapter are relations 
with percents. No comparison or conclusion will appear 
which involves the absolute number of tasks in two levels. 

THE RELATION BETWEEN ALTITUDE AND W ( 1 Oc -f- IOa -f- lOv + 
IOd), I.E., NUMBER OF 40-COMPOSITE CAVD TASKS SUC- 
CEEDED WITH AT A GIVEN LEVEL OF DIFFICULTY 

N individuals are measured, each with, say, a score of 
CAVD composite tasks, each composite being of the same 
difficulty as any other, and each consisting of so many 
single tasks that the correlation between the number right 
in any one composite and the number right in any other is 
perfect. Then any one of the N individuals who succeeds 
with any one of these composites (in the sense of having 50 
percent or more of the single tasks correct) will succeed 
with any other of them ; and the W of any individual will 
be one hundred percent or zero percent. Suppose that the 
same N individuals are measured perfectly in respect of 
altitude of Intellect CAVD. The correlation between alti- 
tude CAVD and W (IOC + lOA + lOV + lOD) will be per- 
fect, every one of the individuals who succeeds with these 
composites having a higher altitude than any one of those 
who fail with them. If each task at a certain level of diffi- 
culty is extensive enough to represent and measure aU of 
CAVD difficulty and no other difficulty — all of CAVD intel- 
lect as it operates with tasks at that level of difficulty and 
nothing but it — then everyone who succeeds with these will 
have a CAVD altitude as high as, or higher than, the alti- 
tude which they represent and no one who falls with them 
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will have a CAVD altitude as high as the altitude which 
they represent. That is, if each task measures all the 
CAVD intellect which can operate at that level and nothing 
hut it, the percent of- tasks mastered at that level will be 
zero or one hundred and will correlate perfectly with alti- 
tude CAVD. 

Stated in another way, any individual who succeeds with 
any task of difficulty d which measures CAVD perfectly as 
it operates at that level of difficulty, will succeed with all 
tasks of less difficulty than d, if these also measure CAVD 
perfectly as it operates at their respective levels of diffi- 
culty; and any individual who fails with any task of diffi- 
culty d will fail with all tasks of greater difficulty than d, 
if these also measure CAVD perfectly as it operates at 
their respective levels. 

These are not axioms necessitated logically by the defi- 
nition of Intellect CAVD and of difficulty CAVD; but con- 
clusions reached by observations of facts. The facts could 
be otherwise. Some men might conceivably succeed with 
tasks like O, P, and Q and fail with tasks like M, N, and O. 

We do not give an absolute empirical proof of these con- 
clusions, because we have not any tasks which measured all 
of the CAVD intellect which operates at any given level of 
difficulty. All the evidence, however, goes to prove their 
truth. 

Kvidence may be found in the correlation between the 
altitude measure and the score of success or failure in 
20-composites (5C -f- 5A + 5V + 5D) corrected for attenu- 
ation, so as to give the correlation between a precise mea- 
sure of altitude and the number of s’s in an examination 
with a very large number of such 20-composites. For ex- 
ample, the average correlation (bi-serial r) of the mea- 
sure of altitude with success in a CAVD 20-composite in the 
case of 98 adult imbeciles was .984 for A, .916 for B, .875 
for C, and .757 for D, averaging .883. The self -cor relation 
of the altitude measure is .94, the inter-correlations of the 
three determinations whose average it is being .92, .77, and 
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.83. The self-correlation (tetrachoric r) of a CAVD 20- 
coinposite in this group is .96 for A, .76 for B, .79 for C, 
and .99 for D, averaging .87^. The correlation between a 
precise measure of altitude and success in 50 percent or 
more of a number of CAVD 20-composites of equal diffi- 


culty may then be expected to be 
.916 _ _ .875 


.984 


V-76 X .94 


for B, 


V-77 X .94 


= for C, and 


V.96 X .94 
.757 


V.99 X .94 


for A, 


for D, 


or, on the average, or .97. 

V.875 X .94 

Also the correlations between altitude and W(1C or lA 
or IV or ID) are very near unity, as will be demonstrated 
in the next section. The correlations between altitude and 
W(10C -f- lOA "h lOV + lOD) a fortiori will be near unity. 

In view of such evidence the conclusions stated in the 
first two pages of this chapter may bo accepted as true. 
There is no reason to expect that the case will be different 
with any fairly catholic form of intellect (such as Picture- 
Completions H- Opposites -f- Geometrical Relations + Rea- 
soning I^roblems of the type devised by Burt + Informa- 
tion; or Analogies + Number-Completions -{- Arithmetical 
Computation + a Common Element test of the type devised 
by Otis) from what it is with CAVD. 


THE RELATION BETWEEN ALTITUDE AND w(lc OB 1a OB Iv OR 
Id), I.E., THE NUMBER OF SINGLE TASKS SUCCEEDED 
WITH AT A GIVEN LEVEL 

This correlation is very close. There are a certain num- 
ber of individuals who are, relaiively to others, much better 
(or worse) in arithmetical tasks than they are in the lin- 
guistic tasks, and whose records prevent perfect correla- 
tion. Also, there are probably other minor specializations 
within Intellect CAVD. But on the whole, individuals 
would be found to follow rather closely the general pattern 
of CAVD intellect shown in Fig. 59 if each of them had 
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been tested with several hundred tasks (one-fourth being 
C; one-fourth. A; one-fourth, V; and one-fourth, D) at 
each level of difficulty from 0 to 44. In general, that is, 
if intellect A has a higher altitude than intellect B, intel- 
lect A will also show a greater W(1C or lA or IV or ID) 
than B at all levels between those w^'here both A and B have 
one hundred percent right and those where both A and B 
have zero percent right; and the amount of superiority of 
A to B in W will be closely similar to the amount of superi- 
ority in altitude. 

To prove this, we have to estimate the relation as it will 
be found with a very large number of single tasks at the 
level of difficulty in question, from data where this number 
is only 40 or less. The evidence is as follows : 

In the case of 237 individuals of group 17, the correla- 
tions between altitude CAVD and percent succeeding in 
tasks N, O, P, and Q were as follow's (P means the Pearson 
r; Sh means the Sheppard r) : 



P 

Sh 

N 

.86 

.76 

0 

.93 

.94 

P 

.91 

.91 

Q 

.81 

.86 

Average 

.88 

.87 


The self-correlations for % s in N, O, P, and Q in this 
group may be taken as approximately .76, using the data 
given in Appendix V, which show that the correlations of 
neighboring 40-composites average .73 in this group. .03 
is added for the effect of the slight remofeiiess. The self- 
correlation of the measure of altitude in this group is com- 
puted as .90 from the intercorrelations of the three inde- 
pendent measures of altitude of which it is the average. 
They are .80, .76, and .71, averaging .757. By the well- 


o/ 

known formula of Spearman, t zwita a will equal I j l 2 ( * 757 ) • 
By this determination, a precise measure of altitude will 

27 
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correlate with a precise measure of ’W(C or A or V or D) 

.875 

to the extent of — - or 1.06. 

V.76 X .90 

As a check on this determination, we have computed the 
obtained correlation between the measure of altitude and 
the sum of the numbers of rights in N, O, P, and Q. It is 
.99. The correlation between a i)recise measure of altitude 
and a precise measure of W(C or A or V or 13) should be 
higher than this obtained correlation. 

In the case of 189 individuals of group 13, the correla- 
tions between altitude CAVP and % s in tasks N, O, P, and 
Q were as follows : 



P 

Sh 

N 

.875 

.84 

O 

.925 

.90i 

P 

.916 

.89 

Q 

.782 

.83 

Average 

.874 

.866 


The average self-correlation for % s in N, O, P, and Q 
in this group may be taken as .74, from the data given in 
Appendix TV. The self-correlation of the measure of alti- 
tude in this group is found by the Spearman formula to be 
.89. The intercorrelations of the three independent mea- 
sures of altitude of which it is the average arc .71, .64, and 
.81. The correlation between altitude and W(C or A or V 
or D), both being measured accurately, will thus be 
87 

\r74X^ ^ 1.07. As a check, we have a correlation of 

.95 between the obtained measure of altitude and the sum 
of the numbers correct in N, O, P, and Q, and a part of M. 

In the case of 246 individuals of group 91, altitude 
CAVD correlates with % s in composites T, J, K, L, and M 
as follows : .58 for I, .82 for J, .92 for K, .82^ for L, and .64^ 
for M (all by the Sh formula). The self-correlations of % s 
in I, J, K, L, and M in this group are respectively .73, .80, 
.74, .86, and .69. The self-correlation of the measure of 
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altitude in this group is .79, the intercorrelation of the 
three measures of which it is the average being .56, .58, 
and .52. It is perhaps unwise to average correlations such 
as these which show wide and regular differences. So we 
correct each for attenuation separately and have, as the 
live resulting determinations of the correlation between 
altitude and W(C or A or V or D), .76, 1.03, 1.20, 1.00, and 
.87. The average of these is .97 ; the median is 1.00. As a 
check we have the correlation between the altitude measure 
and the sum of the numbers correct in I, J, K, L, and M. It 
is .91. 

In the case of 192 individuals of group 9II, altitude 
CAVD correlates with % s in composites K, L, M, and N as 
follows: .73 for K, .90 for L, .91 for M, and .66 for N. The 
average is .80. The self-cor relations of K, L, M, and N are 
respectively .76^, .87 J, .75|, and .75, averaging .80. The 
self-correlation of the measure of altitude in this group may 
be taken as .83, the intereor relations of the three measures 
of which it is the altitude being .50, .635, and .73. So a pre- 
cise measure of altitude will correlate with a precise mea- 
sure of W(C or A or V or D) to the extent of .99 (.91 by K, 
1.07 by L, 1.15 by M, and .84 by N). As a check we have 
a correlation of .96 between the measure of altitude and the 
sum of the numbers right in K, L, M, and N. 

In the ease of 63 university students the correlations 
between altitude CAVD and % s in tasks N, O, P, and Q 
were as follows: 



Sh 

N 

.77 

O 

.92 

P 

.90 

Q 

.70 


The intercorrelations of N, O, P, and Q are : N with O, 
.58 ; O with P, .70 ; and P with Q, .73. The self-correlations 
of N, O, P, and Q may be estimated as .61, .67, .74J, and .76 
by adding .03 to the correlation between neighboring com- 
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I ^ • tll6 measure of altitude in 

this group may be taken as .83, the three measures of which 
it IS the average having intercorrelations of .80, .54, and .50. 
The correlation between a precise measure of altitude and 
a precise measure of width is then computed as 1.08 for N, 
1.23 for 0, 1.14 J for P , and .88 for Q, averaging 1.08. 

As a check on this result, we have the correlation of .98 
between the measure of altitude and the sum of the numbers 
right in N, O, P, 

We have th;;^g' deteriffi^tions of what the correla- 
tion bet^^c;^ altitude CAVD and would be if both were 
measure^^ . , , 

J precisely, namely. 


for group 17 1.06 

“ 13 1.07 

91 .97 

“ 9II .99 


Univ. students 1.06 

with an average of 1.03 =t a mean square error of .019. 

There is an element of insecurity in these corrections 
for attenuation, especially in so far as the self-correlations 
for W(C or A or V or D) are estimated by adding .03 to 
the obtained correlations for neighboring composites. 
However, the empirical correlations between the obtained 
altitude measure and the obtained sum of the W’s (.99, .95, 
.96, .91, and .98) show that the corrected correlations should 
be near unity. 

The same close correlations obtain in groups at low alti- 
tudes. In the ease of the 100 individuals of group im6, the 
correlations of the measure of altitude with % s in C, D, E, 
P, and G, respectively, were .79, .86, .89, .86, and .54, aver- 
aging .79. The self-correlations of the measures of % s 
were, respectively, .80, .86, .84, .83, and .81, averaging .83. 
The self -correlation of the measure of altitude in this group 
is .67 by the Spearman correction, the average intercorrela- 
tion of the three determinations of which it is the average 
being only .407. The correlation between a precise measure 
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of altitude and a precise measure of W(C or A or V or D) 
is then 1.06 bj" this determination. 

The correlation of the obtained measure of altitude with 
the sum of the numbers right in C, D, E, P, and G was .93. 

In the case of the 50 f, the correlation between the ob- 
tained altitude measure and the sum of the number right 
in E, P, G, H, and I was .98. 

In the case of 162 individuals of group 4, the correla- 
tions between altitude CAVD and % s in tasks P, G, H, I, J, 
and K were .48, .83, .93, .95, .75, and .53, respectively. The 
intercorrelations of % s in P, % s in G, and so on, are ; P 
with G = .67; G with H = . 81; H with I = .85^ ; I with J = 
.63 ; J with K = .51. The self-correlations may therefore be 
taken as .70 for P, .77 for G, .86 for H, .77 for I, .60 for J, 
and .54 for K. The self-eor relation of the measure of alti- 
tude is .81, the average intercorrelation of the three mea- 
sures of which it is the average being .59. The most prob- 
able correlation between a precise measure of altitude and 
a precise measure of width is then .64 for P, 1.05 for G, 
1.11^ for H, 1.20 for I, 1.08 for J, and .80 for K, with an 
average of .98. 

As a check on this determination, we have computed the 
correlation between the measure of altitude and the sum of 
the numbers of rights in P, G, H, I, J, and K. It is .96. 

A rough calculation of the correlations for the 180 cases 
of group im3 shows that with them the raw correlations of 
the altitude measure with W (1C or 1 A or IV or ID) in com- 
posites A, B, C, and D will be around .90 and that the cor- 
rected coefficients will be near unity. 

The closeness of these correlations indicates that each 
individual would, if adequately measured by a large num- 
ber of single tasks at each level of difficulty, show a pattern 
closely of the type of Pig. 59. Individuals might be of 
widely different patterns, such as those shown in Pig. 60, 
Pig. 61, and Pig. 62, so that individuals of the same altitude 
would differ widely in width at any level. But, in fact, such 
large divergences in pattern are very scarce in Intellect 
CAVD. 
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Fig. 59 Pig. 60 Pig. 61 Pig. 62 


Pig. 59. The pattern of decrease in percent of single tasks correct with in- 
crease in difficulty, which corresponds to close correlations between 
altitude and W(1C or lA or IV or ID). 

PiOS. 60, 61 and 62. Patterns of decrease in percent of single tasks correct 
with increase in difficulty such as individuals would show 
if the correlations between altitude and W(1C or 
lA or IV or ID) were much below 1.00. 
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How small and scarce they will be in other forms of in- 
tellect, that is, how close a resemblance between altitude 
and width will be found for any other form of intellect, will 
depend upon the constitution of the form in question. In 
CAPIni,, with picture ' completions and information about 
music and art replacing vocabulary and directions tasks, 
the correlations will probably be lower. However, so long 
as the constituents of our composite tasks all concern the 
ability to deal with ideas and symbols for ideas, the amount 
of specialization will be small in comparison with the total 
variation in ability, so that the correlations will be high. 

THE RELATION BETWEEN ALTITUDE AND AREA OF INTELLECT 

The facts brought forward in the first and second sec- 
tions of this chapter prove that the A (IOC -h lOA + lOV -|- 
lOD) of any intellect and the altitude of that intellect are 
determined almost or quite entirely by the same cause or 
causes. 

The facts of the third and fourth sections prove that to 
a very considerable extent this is true for the A (1C or lA 
or IV or ID) of any intellect and its altitude. A verifica- 
tion of this by the direct measurement of A(1C or lA or IV 
or ID) is not yet possible because the number of tasks at 
each level of difficulty is unknown. Indirectly, it may be 
partially verified as follows: If n single tasks are taken 
from each level from zero to forty-five, one-fourth being C, 
one-fourth A, one-fourth V, and one-fourth D, and individ- 
uals are measured in respect of these, n being sufficiently 
large, the A’s so obtained will have the same rank as A^s 
obtained by an examination where the intellects are tested 
with all tasks at all levels. The area for the selection of 
n at 0, + n at 1, n at 2, -f- n at 3, and so on, may be taken 
to be approximately the area found by assuming that each 
intellect will succeed with all or nearly all of the single ele- 
ments at levels below the highest level where it obtains 100 
percent right and will fail with all or nearly all of the single 
elements at levels above the lowest level where it obtains 
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zero percent right (or only that percent which mere chance 
guessing could give). 

By permitting some estimating of scores, this procedure 
may he carried out. The results appear in Fig. 63. The 
cases entered in Fig. 63 are all taken at random so far as 



Fio. 63. The relation between CAVD altitude and area in a wATwpUTig of 
comprising N tasks for each unit of altitude. 
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the relation in question is concerned. Those used were all 
that had 37 or more right in the easiest altitude with which 
they were tested, or a random selection from all such. The 
groups used were im3, f, 4, 91, 17 and the group of 63 univer- 
sity students. The area number was computed as follows: 
I. Assume that, at each unit of altitude up to the easiest 
altitude at which the person was measured, he had 40 (i.e., 
all) right. JI. Count the number he had right over the 
range at which he was measured; and estimate from this 
how many he would have had right had he been tested with 
40 single tasks at each unit of altitude over this range. III. 
Estimate the number which he would have had right at all 
altitudes above the highest at which he was tested, using 
arbitrarily the number which he had right at the highest 
altitude at which he was tested. The area number is the 
sum of the three numbers obtained by I, II, and III. 

The area number thus ranges possibly from 957 for an 
im3 who had 37 right in Composite A and none right in any 
higher composite, to 1,800 for a person who had 40 right in 
N and also in O, P, and Q. The lowest actual area number 
among the cases used was 1,063; the highest was 1,760. 
The very close interdependence of area and altitude shown 
by Fig. 63 would be little if at all reduced if more extensive 
and precise measures were available.' 

There is thus a high degree of genuine unity to Intellect 
CAVD, not assumed but discovered. We began with a mea- 
surement in the form of an inventory, differing from a bare 
enumeration of success or failure with actual tasks only in 
that the tasks were graded in difl&culty. We end with mea- 
surements of altitude, width and area which intercorrelate 
so closely that they may reasonably be treated as results of 
a closely knit set of causes. Whatever makes one intellect 
able to do much harder CAVD tasks than another intellect 

1 It would be reduced inasmuch as some of the errors now involved act in 
the same direction on the altitude measurement and on the area measurement. 
It would be increased inasmuch as the purely chance fraction of the error acts 
to reduce the correlation. 
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also makes it able to do many more tasks than that other 
can do. After the necessary data have been collected, width 
at any altitude, and so total area, will be predictable in the 
case of Intellect CAVB (and presumably in the case of 
other forms of intellect) rather precisely from altitude 
alone. 

THE RELATION OF ALTITUDE OR LEVEL OF INTELLECT TO SPEED 

It is important to know the relation between level and 
speed for two reasons. If the relation is very close, the 
speed of performing tasks which all can perform would be 
an admirable practical measure of intellect. The record 
would be in time, an unimpeachable and most convenient 
unit. If, on the other hand, the correlation is very low, the 
practice of giving credit for speed in group examinations 
should probably be amended. 

Dr. Hunsicker [’25] has made extensive individual mea- 
surements upon 82 adults and 81 school children, taking the 
time for easy problems in arithmetic and for easy comple- 
tions, such as appear in our composites E, F, and G; and 
then testing the person with harder and harder tasks until 
the level was reached where he could not obtain fifty per- 
cent right. 

The correlations which she obtained between altitude 
and rate (the reciprocal of the time required for tasks done 
with no, or very, very few errors) are shown in Table 130. 
They are much too low to make it advisable to use peed 
at easy tasks as a measure of the altitude or ^ ^ea 

of intellect, except possibly in the case where tl 
able for the examination is very short. They a 
low that it seems unwise to attach much weight .o speed in 
intelligence examinations in general.* A graded or ladder 
test of thirty minutes containing 5 levels each consisting of 
ten words and five arithmetical problems* using small num- 

2 Except, of course, in the case of tests (such as the substitution test) 
'where speed measures the speed of learning* 

«Or containing ten opposites and ten questions of arithmetical informa- 
tion^ or containing five directions and five arithmetical problema. 
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bers, will in all probability show a closer correlation with 
any reasonable criterion of intellect than will a thirty unin- 
ntes' speed test. 


l^ABLE 130. 

CORREIiATIONS, BAW AND CORRECTED FOR ATTENUATION, BETWEEN BATE AND 

Level. (After Hunsiceer, '25, Table V.) 


Individual 

Testing 

No. in 
group 

Arithmetic 

Baw Corrected 

r T 

Completion 

Baw Corrected 

r r 

Average 

w. c. 

28 

.29 

.35 

.50 

.56 

.46 

S. C. 

54 

.46 

.55 

.19 

.23 

.39 

P. S. 189 . 

32 

.49 

.58 

.49 

.64 

.61 

P. S. 6 . 

49 

.29 

.35 

.41 

.50 

.43 

Average 



.46 


.48 

.47 


We have extended Dr. Hunsieker^s work by a measure- 
ment of the speed of doing a collection of CAVD tasks 
chosen from levels I and below in the case of 63 university 
students for ^hom a measure of CAVD altitude was ob- 
tained by the use of composites N, O, P, Q, and a still harder 
composite. 

There were some errors in the easy tasks, so we have 
computed r.,.®, the partial correlation between speed and 
altitude, for those making equal numbers of errors in the 
rate test. 

r,. = + .403 r»e = — .084 Ta, == — .484, 

hence r8a.e = .416. 

The self-correlation of the measure of altitude is .83 for 
this group; the self-correlation of the measure of speed is 
not known but is almost certainly between .7 and .9. If the 
.403 were corrected for attenuation, the result for CAVD 
would thus be fairly close to Dr. Hunsicker’s results for A 
and C. 




CHAPTER XIV 


The Meaning of Scores Obtained in Standard 

iNTEtiilGENCE EXAMINATIONS 
THE MEANING OF THE BINET MENTAL AGE 

A Binet Mental age is a rough measure of relative alti- 
tude A D Inf Ot, using Ot to mean “other tasks found or 
alleged to deserve inclusion in a battery to measure intelli- 
gence”; or, more exactly, of the relative A (la or Id or linf 
or lot) of a sampling of a certain number of tasks at each 
of certain levels. This A will correlate closely with alti- 
tude. Up to about M. A. 14, Binet scores are defined by the 
probable median or average chronological age of those who 
would obtain such a score, in the group by which the ex- 
amination was standardized. Above M. A. 14, the scores 
are arbitrary. 

Until tlie numbers of tasks at each level of difficulty are 
known, and perhaps even after they arc known, a Binet 
Mental Age may best be treated as a measure of altitude — 
of how hard tasks the person can succeed with. If this is 
done, nothing will be lost from sound present uses and cer- 
tain misapprehensions will be avoided. For example, 
everyone will understand that a very small increment of 
mental age at the high ages may mean a very largo incre- 
ment in area of intellect or percentage of success with the 
total mass of intellectual tasks which life may offer, and 
that a very large increment of mental age at the low ages 
may mean a relatively small increase in the total number of 
tasks achievable or in the total number of connections 
formed. 

The great merit of the Binet Test is that it is a graded 
scale for intellectual difficulty, and it is only weakened by 
being interpreted loosely as a measure of some mysterious 
essence called intelligence which grows in man. The weak- 
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ening is not disastrc^us simply because^ as was shown in the 
previous chapter, altitude and width (and consequently 
area) of intellect aire so closely correlated. 

Miss Rowell is jtn ensuring the values of Stanford Binet 
M. A. 10, M. A. 11, M. A.' 12, etc., in terms of the absolute 
units of the CAVll scale in so far as one can be said to mea- 
sure the equivale-nce of two series of magnitudes which may 
not be measures of exactly the same fact in nature, and of 
which one (tl^e Binet) may not measure varying amounts 
of the same fact. We have found that adults of Shinford 
Binet Men^^al Age 48 months, or 4 years, will show an alti- 
tude of' about 26 in Intellect CAVD; and that adults of 
Stanford Binet Mental Age 78 months, or 6^ years, will 
shov/ an altitude of about 30 in Intellect CAVD. When, by 
these measurements or by others, the differences in the 
M. A. scores are put in equal units and referred to the abso- 
l/dte zero of intellectual difficulty as located by us, or as 
more accurately located by future workers, the Binet scale 
and measurements will have a much greater value than they 
now have. 

What has been said of the Binet applies equally to the 
Herring Kxaminatioii, which is an alternative Binet. 

THE MEANING OF SCORES OBTAINED IN STANDARD GROUP 

EXAMINATIONS 

The significance of scores in group tests such as the 
Army Alpha, National, Otis, may best first be considered 
with disregard of the factor of speed; that is, on the as- 
sumption that the scores of individuals represent what 
they can do with time enough allowed to exhaust t)ieir 
abilities. 

The score does not measure either altitude or widtude^ 
area of intellect. It does not measure altitude, becau&rely 
number of tasks between levels equally far apart is m cor- 
essarily the same. It does not measure width, beca:etions 
score is not divided up into a number of sub-scorted for 
representing the number of successes at a certain ing alti- 
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does not measure area, because it meastires neither altitude 
nor width, and because the percent which the tasks are of 
those that might be had at any level of difficulty is not 
known. 

Although one of these group tests doe-s not in a rigorous 
sense measure any one of the three, the stiore in it is about 
as closely symptomatic of altitude as the Score in any test 
requiring so short a time could be. It is also closely symp- 
tomatic of the average width of intellect at and near the 
levels of difficulty represented by its tasks. One of these 
group examinations is in fact very much like wh«t we have 
when we put together five or six of our CAVD 40-com- 
posites that are in a sequence for difficulty. The difiLorence 
between a set of these CAVD composites from about G -o N 
and Army Alpha or the National or the Otis (no time liinit 
being set) is that in the case of the CAVD composites, 
know how many single tasks there are at each level of diffi- 
culty, and we know how far apart the levels of difficulty arc, 
and we can not only make a summation of credits, but also 
can make an altitude score, and a width score at each level. 
In Army Alpha or the National or the Otis, the total sum- 
mation score is not susceptible of such an analysis. 

Except for the speed element, then, one of these stock 
intelligence examinations may be regarded as a series of 
composites unequal in the number of their elements, and 
undefined as to the distances between levels. The addition 
of the speed clement complicates matters and theoretically 
makes the significance of the score incapable of interpreta- 
tion except in terms of what people of a certain sort do in 
IL that kind of a test when it is scored in that way. 
tat Practically, however, the speed element does not make 
formed.’Coros in these examinations, as they are administered 
The g case of most of the individuals who are measured by 
scale for very much different in significance from the scores 
being inteFould be obtained with no time limits set. A fev 
essence caLare nervously upset by the instructions to work at 
ley can ; a few cautious, critical workers do not have 
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time enough to do as many of the hard tasks as they are 
really able to do; a few persons are scored unduly high 
because they utilize the time especially shrewdly, while a 
few others are scored unduly low because they dally too 
long over tasks at which they fail, or leave tasks undone 
which the use of a little more time would have enabled them 
to finish. But, in general, the scores in these speed tests 
correlate very closely with the scores obtained when a 
longer time allowance is given, partly because the correla- 
tion between speed and altitude is positive, but more be- 
cause the standard time allowance is long enough to enable 
most of the candidates to do most of the tasks which they 
could under any circumstances do. 

The experiments of the Army psychologists on the re- 
sult of doubling the time allowance for the Alpha and Beta 
examinations are well known [Memoirs, ’21, pages 415- 
4201. The general result was that there was a slight im- 
provement in the correlation with officers’ ratings for in- 
telligence, and a close correlation between the score in 
single time and the score in double time (r = .967), which is 
probably as high as the self-correlation of the determina- 
tions would permit. 

Dr. J. R. Clark has investigated the influence of alti- 
tude and of speed upon the abilities measured by the Stan- 
ford Binet, the Otis Self- Administering Test, and the Ter- 
man Group Test, in the case of school pupils from Grades 
7 to 12. 

His results are not entirely clear, because his measures 
of speed are afflicted by rather large variable errors, and 
are perhaps also disturbed by the presence of an altitude 
factor; but on the whole they indicate that scores in these 
stock examinations are determined much more by altitude 
than by speed, and perhaps are determined almost entirely 
by altitude and width. The average of the six speed cor- 
relations (speed in arithmetic and speed in completions 
with Binet, Otis, and Terman) each being corrected for 
attentuation, is .54. The average of the corresponding alti- 



406 


THE MEASUBEMEHT OF IHTELUGEHCE 


tilde correlations is .70. The average of the four correla- 
tions between speed and altitude is not given, nor all the 
data whence to obtain it. Ar speed with Ar altitude (cor- 
rected r) is .76; Co speed with Co altitude is .40; the aver- 
age is thus .58. The other two r’s are not given. They 
would presumably be lower. If their average is estimated, 
we can compute the partial correlations of speed with Binet, 
Otis, and Tcrman for persons of equal altitude in Ar or Co 
and of altitude with Binet, Otis, and Terman for persons of 
equal speed in Ar or Co. Estimating this average as .48, 
the partial correlations are .28 for speed and .58 for alti- 
tude. A more instructive set of measurements is of the 
relations between speed in general and altitude in general 
to scores in Binet in general, Otis in general, and Terman 
in general.' These I>r. Clark has made. He finds that dif- 
ferences amongst individuals in the score in one of these 
examinations are almost perfectly correlated with differ- 
ences in what is common to their two altitudes, and much 
less closely eorrelaled with differences in what is common 
to their two speeds. We quote his results. 




general level and Binet 


aI 


(r 


ar. level and Binet 


)(r 


CO. level and Binet . 


(r 


ar, level and co. Icve 


.)(r 


Binet and Bluet i 


.65 X .65 


(.55) (.90*) 
Similarly 

F general level and Otla 


.93. 


^ aI 


(r 


ar, level and Ot 


Is) (l* 


CO, level and Ot 


is) 


(r 


ar. level and co. level 


)(r 


Otla and Oti8> 


.83 X .61 
(.55) (.90*) 


.98, 


and 


general level and Terman 


1 * * Binet in general * ^ means the average seore in an infinite number of 
teats patterned after the Stanford Binet. 

* Estimated. 
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(r 


ar. level and Terraan 


)(r 


CO. level and TermanJ 


(.55) (.90*) 


_ ^ .80X.66 _ 


Approx. 1. 


(.55) (.90*) 

In the same way, the relationship between ‘general 
speed’ scores and intelligence test scores is found to be: 


genera 1 speed aud Blnet 


general speed and Otis 


general speed and Terman 




55 X .49 


50) (.90*) 
71 X .49 


-= .59. 


50) (.90*) 


= .77. 


67 X .32 


(.50) (.90*) 


.49. 


The mean of these correlations is .62.” — [Clark, ’25, p. 
33f.] 

If partial correlations are computed using .97, .62, and 
any reasonable estimate for the intereorrelation of speed 
in general with altitude in general, they are very high for 
altitude and very low for speed. For example, let r for 
speed in general with altitude in general in this group be 
.65. We then have .95 and .00. Letting it be .60, we have 
.95 and .20. Letting it be .70, we have .96 and — .34. 

Even with time limits, then, the scores in standard 
group examinations may properly be treated approximately 
as summati(>n-of-credit scores of tlie same fundamental 
nature as a Stanford-TBinet smnmation score or as a CAVD 
summation score. The chief difference for practical pur- 
poses is that the Stanford-Binet summation is of rights in a 
series of tasks specified as to difficulty, six for the interval 
from Chr. Age 3.0 to Chr. Age 4.0; six for the interval 
from Chr. Age 4.0 to Chr. Age 5.0; and so on; and the 
CAVD summation score is of tasks specified as Diff. 23, 
Diff. 26J, Diif. 28 J, and so on ; whereas the Army Alpha or 
National or Terman Group summations are from an unde- 
fined collection of tasks. 


* Estimated. 


28 
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We may expect that, in the future, all these group ex- 
aminations which have proved themselves so convenient in 
getting quickly and cheaply an approximate measure of 
something which is reasonably called “intelligence,’’ will 
retain these advantages and gain those of clearer interpre- 
tation by certain changes in the method of construction 
which are recommended by the princii>les which we have 
formulated. Instead of being a collection of small tasks of 
undefined location as to difficulty, they will be made in 
levels with a definite number in each level, and the levels 
will be placed, at least approximately, equally far apart. 
A more liberal time allowance will be given, and each indi- 
vidual who is examined will be instructed to take as much 
time as he needs and to go as far as he can. The group test 
can then be scored by a summation of credits just as now, 
but that summation will have resulted from a combination 
of scores of the number right at each of various levels of 
known difficulty. An altitude score can be inferred from it, 
since its correlation with altitude will be nearly as high as 
its own self-correlation permits. Or an altitude score can 
be computed more directly by some such procedure as was 
described in the case of CAVD in Chapter II. 

THE MEANING OF SCORES OBTAINED IN TESTS OF THE ABILITY TO 

LEARN AND TO IMPROVE 

The only test of ability to learn which has been widely 
used as a measure of intellect is the substitution test.* 
There has been far too little experimentation with ability 
to learn as a test of intellect. The early work seemed dis- 
couraging, the correlations with a criterion being appar- 
ently much lower per unit of time in testing and labor in 
scoring for tests of improvement than for tests of status. 
We have elsewhere shown [Thorndike, ’24] that this may 
be due in large measure to the peculiar action of the error 

2 The form used in the National is typical. The amount done correctly in 
a given time depends largely on bow quickly and accurately the individual 
learns the key. 
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of measurement upon the relation between initial ability 
and gain. Thomson [^24] has provided the requisite cor- 
rection formula. While the recognition of this error and 
the correction for its influence on the correlation does not 
improve the actual diagnostic or predictive value of a short 
learning test, it does demonstrate that a test which is long 
enough to measure improvability accurately may have a 
much greater diagnostic and predictive value than had 
seemed possible. Another difficulty which has discouraged 
experimentation with tests of improvability is the lack of 
any sound general theory for comparing gains from differ- 
ent starting points ; and this may greatly limit the scope of 
such tests. On the other hand, measures of improvability 
have exceptional advantages in respect of universality of 
application and freedom from improper training. They 
may also be much less susceptible to environmental differ- 
ences than are the ordinary measures of status. 

An investigation of the altitude of learning, that is, of 
how hard things a person can learn to do ; and of the speed 
of learning certain things which all or nearly all can learn 
if they have time enough, is very much needed. Until such 
an investigation is made, the sound procedure with scores 
in substitution tests and the like is to treat the varying 
scores as representing success with tasks varying in diffi- 
culty. A score of K -f- 2 thus means success with a harder 
task than does a score of K -f- 1, and so on. The differ- 
ences in difficulty corresponding to the differences in the 
scores can then be measured by the principles and tech- 
niques presented in this volume. 

There are two flnal matters which concern all existing 
tests and scales for intellect. The first is the matter of the 
selection of the tasks. The second is the matter of the re- 
liability of the measure. 

In all of them the selection of the single tasks has been 
narrow and more or less arbitrary. Sinet chose tasks 
which older children did and younger children could not do. 
In the National the choice was made in view of a criterion 
consisting of grade reached in school, intelligence as esti- 
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mated by teachers, and the like. A general fund of knowl- 
edge that such and such tasks are allied to various symp- 
toms and assumed criteria of intelligence is more or less 
wisely used. In none of them has the selection been made 
so as to represent or sample in any defined way any total 
series of tasks which the authors of the examination re- 
garded as being the totality of intellectual tasks. 

This is not a serious defect for the ordinary purposes 
of ranking individuals according to that ill-defined trait 
known as inlelligence, for two reasons. F^irst, most of the 
examinations in common use are made up of verbal and 
numerical and factual tasks in somewhat the same propor- 
tions; and, second, the intercorrelations of different sorts 
of tasks, so long as all concern the ability to manage ideas, 
are all fairly close. 

However, there is no loss and a considerable gain, espe- 
cially for rigorous treatment, if a definite plan for the selec- 
tion of tasks is used, as in the CAVD series, so that the 
nature of the fact measured is clearly defined. If the Army 
Alpha, for example, were called D, A, Cs, So, Di, Ac, An, 
Inf, and constructed so that there were three of each of 
these sorts of tasks at each of eight levels, it would be a 
more useful instrument. 

In respect of the reliability of the determination, our 
examinations have been far too lax, especially at the high 
levels, to serve well for scientific purposes or for such prac- 
tical purposes as require any considerable exactitude. The 
meaning of any score is obscured by the fact that so large a 
portion of it is chance error. At the high levels a small 
error in altitude may mean a very large error in area. The 
neglect of this matter has indeed been almost scandalous, 
since in the case of many widely used intelligence examina- 
tions, the amount of the chance error is not even known. 

In connection with our inquiry. Miss Woodyard is con- 
ducting a careful investigation of the nature and amount 
and causation of the chance error in mental measurements, 
which will be treated in a separate report. We present here 
only the facts concerning the reliability of a determination 
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of CAVD altit'ade such as results from a measurement of 
an individual with four 40-eomposite tasks near his level 
of difficulty. To obtain such a measurement in an indi- 
vidual examination •will require about thirty minutes of 
exploratory testing and about two hours with the four com- 
posites. To obtain it in a group examination -will require 
usually that the group be tested with five 40-composites re- 
quiring from three hours at levels H to L, up to five hours 
at levels N to R. 

The mean square error of an altitude determined from 
the three successive 40-composite scores whose sum is near- 
est to 60 is as shown below for various groups. 

MEAN SQUARE ERROR OF A CAVD ALTITUDE IN UNITS OF THE CAVD 

SCALE (1.00 EQUALLING Oig) 

The median error is reported for each group. 


100 adults of mental age 2^ to 4^ .41 

100 adults of mental age 6 .32 

115 pupils in Grade 4 .40 

100 pupils in Grade 9 .29 

63 university students .37^ 


The mean square error is thus about .35, or about one 
fiftieth of the difference between a low grade imbecile of 
mental age 3 years and a very gifted adult, one person in a 
thousand. 

By improving the CAVD composites, the error doubt- 
less can be reduced somewhat. Also, further investigation 
may disclose a procedure more serviceable than the “60’^ 
rule (described in Chapter XI, p. 369), and not more com- 
plicated or time-consuming. But the decreases will prob- 
ably be small- In the main, decrease in the error must be 
attained by increase in the time of the examination. 

The existing stock intelligence examinations may be 
superior to CAVD in this respect, but the probability is 
that they are inferior to it. It has the advantage of build- 
ing upon the results gained by them, and of choosing from 
the best task-material known to date. 



CHAPTER XV 


The Nature of iNTELiiEor 

We have learned to think of intellect as the ability to 
succeed with intellectual tasks, and to measure it by mak- 
ing an inventory of a fair sampling from these tasks, ar- 
ranging these in levels of intellectual difficulty, and observ- 
ing how many the intellect in question succeeds with at 
each level (and, if we wish, how long a time each success 
takes). From this graded inventory, we may compute mea- 
sures of altitude or level, of width or range or extent at 
each level, and of total area. For Intellect CAVD, the pat- 
tern for a sample with n tasks at each level, if n is large 
enough, is very similar for all individuals of roughly simi- 
lar training. Consequently, the altitude, the total area, and 
the width at any level are closely interdependent. 

Any defined intellect can be treated as we have treated 
Intellect CAVD. 

Such a definition in terms of tasks accomplishable, and 
such a measurement in terms of the contents of a graded 
inventory is sound and useful, but is not entirely satisfying. 
One cherishes the hope that some simpler, more unitary 
fact exists as the cause of intellect and that variations in 
the magnitude of this fact may provide a single funda- 
mental scale which will account for levels and range and 
surface. Moreover, one realizes the desirability of search 
for the physiological cause of intellect, regardless of 
whether that cause be single and simple or manifold and 
complex. 

Our consideration of these matters has led us to a hy- 
pothesis concerning the nature and causation of intellect 
for which we have found fairly substantial evidence. It is 
the purpose of this chapter to present and discuss this 
hypothesis. 
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A WORKING DEFINITION OF INTELLECT 

Since this hypothesis concerns intellect in general as 
well as any defined segment of it such as Intellect CAVD, 
we may revert to vaguer and more catholic conceptions. 
The fact of human life of whose nature we seek a more 
exact description is the ability to deal with things or per- 
sons or ideas by the use of ideas. We contrast intellectual 
power over things, as by ideas about length or weight or 
heat, with non-intellectual power over things, as by strength 
or skill or acuity of vision. We contrast intellectual power 
over people, as by consideration of facts about them, with 
non-intellectual power over them, as by good temper or 
courage or physical charm. We contrast intellectual power 
over ideas, as by using other ideas to gain success with 
them, with non-intellectual power over them, as by industry 
or patience. 

The facts and arguments which we shall present do not, 
however, depend for relevance or value upon the acceptance 
of this particular identification and demarcation of intel- 
lect. They will apply nearly or quite as well to any prelim- 
inary description which any competent psychologist would 
devise for those features of life wherein the Aristotles dif- 
fer most widely from the inmates of asylums for idiots, and 
wherein the life of a man thinking effectively about mathe- 
matics or medicine or manufacturing differs from the life 
the same man eating, drinking, swimming or playing tennis 
without, as we say, an idea in his head. 

If a score of competent psychologists should list on the 
one hand all the products whose production depends pri- 
marily upon intellect — all the tasks for success with which 
intellect is the sine qua non] and on the other all the prod- 
ucts or tasks which they regarded as non-intellectual — suc- 
cess thereat being independent of intellect — they would 
show very substantial agreement. Where they appeared to 
differ, the differences would be unimportant for our pur- 
pose. Very seldom would the same task appear on opposite 
sides of the ledger. When it did, the difference would re- 
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solve itself into a difference in favor of a narrower restric- 
tion of intellect (for example, to ability to deal with ab- 
stract ideas or to ability to deal with relations), or in favor 
of a wider extension of it (for example, to certain tasks 
where ideas are not at work, at least not obviously) . The 
facts and arguments which we shall present will apply re- 
gardless of such shrinkage or swelling in the area regarded 
as intellectual. 

Intellect may be CAVD, or CAVD plus ability in giving 
the opposites of words, making it GAVDO ; or that, plus in- 
sight into spatial relations, making it CAVDOS; or that 
plus ability in inductive and deductive reasonings, making 
it CAVDOSR, and so on. 

Beginning, then, with this loosely determined group of 
products which intellect produces, tasks at which intellect 
brings success, we may inquire concerning its observable or 
surface nature as a fact in human behavior or its deeper 
nature as a fact of fundamental processes in the mind or 
brain. We may investigate the thinldng and action of men 
who have much intellect to discover more precisely and 
fully the features in which they differ from men who have 
little; or we may try to discover more ultimate causes of 
these differences. We may compare a man^s obvious life, 
when he is using his intellect little or not at all, with his life 
when he makes large use of it, other factors remaining the 
same, to see just what the differences are ; or we may try to 
discover hidden forces which produce these differences. 
We may study the nature of intellectual tasks, the produc- 
tion of intellectual products, or the nature of the ultimate 
power or powers whereby a man can succeed with such 
tasks. There may be, of course, much to be revealed con- 
cerning facts intermediate between the description of in- 
tellectual tasks and the discovery of their ultimate cause. 

The standard orthodox view of the surface nature of 
intellect has been that it is divided rather sharply into a 
lower half, mere connection-forming or the association of 
ideas, which acquires information and specialized habits of 
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thin ki ng ; and a higher half characterized by abstraction, 
generalization, the perception and nse of relations and the 
selection and control of habits in inference or reasoning, 
and ability to manage novel or original tasks. The ortho- 
dox view of its deeper' nature, so far as this has received 
attention, has been that the mere connection or association 
of ideas depends upon the physiological mechanism whereby 
a nerve stimulus is conducted to and excites action in neu- 
rones A, B, C, rather than any others, but that the higher 
processes depend upon something quite different. There 
would be little agreement as to what this something was, 
indeed little effort to think or imagine what it could be, but 
there would be much confidence that it was not the mecha- 
nism of habit formation. 

THE HYPOTHESIS THAT QUALITY OF INTELLECT DEPENDS UPON 
QUANTITY OF CONNECTIONS 

The hypothesis which we present and shall defend ad- 
mits the distinction in respect of surface behavior, but as- 
serts that in their deeper nature the higher forms of intel- 
lectual operation are identical with mere association or 
connection forming, depending upon the same sort of physi- 
ological connections but requiring many more of them. By 
the same argument the person whose intellect is greater or 
higher or better than that of another person differs from 
him in the last anal3’^sis in having, not a new sort of physio- 
logical process, but simply a larger number of connections 
of the ordinary sort. 

More exactlj" our h^rpothesis is as follows: Let c repre- 
sent whatever anatomical and physiological fact corre- 
sponds to the possibility of forming one connection or asso- 
ciation or bond between an idea or any part or aspect or 
feature thereof and a sequent idea or movement or any part 
or aspect or feature thereof. Then if individuals /i, Jg, /», 
74, etc., differing in the number of c’s which they possess 
but alike in other respects, are subjected to identical en- 
vironments, the amount or degree of intellect which any 
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one of them manifests, and the extent to which he mani- 
fests ** higher intellectual processes than the other indi- 
viduals, will be closely proportional to the number of c’s 
which he possesses. If we ranlc them by intelligence-exami- 
nation scores, the order will be that of the number of c's. 
If we rank intellectual processes in a scale from lower, such 
as mere information, to higher, such as reasoning, the indi- 
viduals who manifest the highest processes will have the 
largest number of c’s. 

The view of common sense, of educational science, and 
of those who have constructed tests for measuring intellect, 
has been that intellect is a power to respond correctly^ that 
the quality of the responses is a primary criterion of the 
degree of intellect. The teacher and the test-maker would 
insist that correct judgments and valid inferences required 
more intellect than the wrong judgments and faulty in- 
ferences. 

The hypothesis which we present accepts this view, but 
makes a sharp distinction in this regard between the orig- 
inal intellectual capacity which a man has and the actual 
intellectual products which he produces. It credits the 
quality of the ideas that a man acquires, and the truth or 
falsity of the judgments which he makes, and, to some ex- 
tent, even the validity of the inferences which he draws 
from any given data, largely to his training. The average 
man today has better ideas about lightning than Aristotle 
had, can make more correct judgments about eclipses than 
Moses could, and, if trained in science, may well draw more 
valid inferences from observing the action of acids on 
metals than either Aristotle or Moses did. With approxi- 
mate equality of training, the quality of intellectual re- 
sponses is an essential index of intellectual capacity, but it 
may be deceptive if the inequalities are great. 

Our hypothesis limits itself to the original capacity. If 
by original nature, apart from all training, a man possesses 
tendencies to be right rather than wrong in his judgments, 
to hold true rather than false ideas, to make justifiable 
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rather than unjustifiable inferences, more or less than other 
men, in so far forth those tendencies are due to his having 
more or fewer c*s than other men. 

The essential element of our hypothesis is that it offers 
a purely quantitative fact, the niunber of c’s, as the cause 
of qualitative differences either in the kind of operation 
{e.g.y association versus reasoning) or in the quality of the 
result obtained (e.g., truth versus error, wisdom versus 
folly), so far as these qualitative differences are caused by 
original nature. 

We need to make clearer what is meant by “one connec- 
tion or association or bond between an idea or any part or 
aspect or feature thereof, or a group of ideas and a sequent 
idea or movement or any part or feature or aspect thereof.” 
By “connection or association or bond between a and b,” is 
meant the probability or certainty that if a occurs in a per- 
son, h will occur in him shortly thereafter (say within a 
second) unless some counteracting force prevents. For the 
sake of simplicity, we may think of all cases as cases of cer- 
tainty. The existence of the connection a-^h then means 
that whenever a occurs, h will follow within a very short 
time interval, unless restrained by some contrary force. 
Thus in a child who has learned the multiplication table, the 
idea 2 times 5 wdll always be followed by the idea 10, unless 
some contrary force jirevents. 

The h which follow's a may be the suppression or pre- 
vention of an idea or movement as truly as its appearance. 
Connections where a inhibits some event may indeed well 
be as numerous as connections where a releases or produces 
some event. The h may also be one step toAvard an event or 
one partial condition of it so that, for example, ax may be 
followed by hx with no obvious difference in the person, and 
fl 2 may be followed by again with no observable differ- 
ence in the person, but if ax and Uo act closely together in 
time, an obvious difference c in the person may always fol- 
low. Similarly the h may be one step toward or one partial 
condition of the inhibition of some occurrence, so that, for 
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example, at alone may not prevent c, and a2 alone may not 
prevent c, but Ui + <X2 in close enough temporal proximity 
may entirely prevent c from occurring. 

By an *‘idea’’ is meant any small portion of the stream 
of thought, such as a precept, or an image, or a meaning, 
or a word of inner speech.^ It is intended to omit unorgan- 
ized sensory stuff such as the person does not ever isolate 
or identify or name, and emotional stuff such as excitement, 
irritabilitj'^, or fear. But no sharp lines need be drawn, for 
the hypothesis will remain valid even if “ideas” are inter- 
preted loosely to include more than we have in mind; and 
the hypothesis will be fruitful even if “ideas” are inter- 
preted very, very narrowly, say to include only words. It 
is not intended to beg the question of consciousness or 
bodily action by the use of the word image. Whether 
“ideas” be envisaged as facts of conscious awareness or 
as facts of bodily behavior is a matter of indifference to the 
hypothesis. 

There remains to bo clarified the apparently innocent 
and unambiguous word “one,” which really conceals a nest 
of difficulties. We think readily about “one idea,” “part 
of an idea,” “many ideas, “two images that are nearly 
alike,” “ten thousand percepts,” and the like; and the 
treatment of percepts, images, and relations as separable 
and capable of enumeration is doubtless useful and in a cer- 
tain correspondence with reality. Yet it is hard to decide 
when and why the reader’s percept of, say, the word 
** Adam” shall be counted as one percept of a word, or four 
percepts each of one letter, or ten percepts each of one line ; 
or when and why his visual image of a square shall be 
counted as one image of a square, or as eight images of 
lines and angles, or as a fraction of a total visual image of 
a square on a certain background. Naive common-sense 
calls “Two and three are five” one idea, and with a certain 

1 BensoTi'inotor connectionB, buc^ as are formed in learning to swim, dive, 
box, or wrestle, and moral or ter^peramental connections, such, as are formed 
in keeping one’s temper, or being courageous, are thus considered as, at least 
partially, outside of intellect 
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suitability, but that sort of a *‘*‘one** is obviously different 
from the *‘'one^* used when a person has the idea of two, 
or the idea of three, or the idea of “and-ness.’* 

In strictness the entire status of the mind (or of the 
central nervous system) at any instant connects with or 
leads to its entire status at the next instant, and any seg- 
regation of a part of this total status as one idea, and any 
attribution of a part of the sequent status to this idea’s 
associative potency, is an incomplete statement of what 
happens. 

These difficulties and others of like nature may best be 
met, not by trying to set up rigorous criteria for what shall 
be one idea, what a part of an idea, and what a group of 
ideas, nor, on the other hand, by a refusal to use the anal- 
yses which common sense and science have found profitable, 
but rather by realizing that ideas are not like eggs in a 
basket nor like eggs in an omelet, but are what they are. If 
we must liken them to something, let it be tones in a sym- 
phony or factions in a party or neurones in the brain, or 
some other ease of a very complex organization where sci- 
ence can and should separate the total into parts, but 'where 
these parts are siilitting and combining from time to time 
and are being influenced in their action by more or less of 
the total organization. 

We must, of course, be consistent, in any comparative 
enumerations, not calling the same fact now one idea, now 
two, and now ten, to tlio prejudice of the truth. 

The next matter to note is that there may be associa- 
tions or bonds from different ideas, from a to h, from a to 
c, from a to d, and so on, according to slightly different con- 
ditions in the general status of which o is a part. Thus the 
idea of “12” may call up “11 + 1,” or “10 + 2,” or “4X 
3,” or “6 X2,” or “dozen,” or “not prime,” or ‘*a num- 
ber,” or “XII,” according to relatively minor conditions 
attached to “12.” There may likewise be connections from 
many different ideas to the same idea, or to the same move- 
ment, as when we think “yes” or nod the head to a hundred 
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different questions. This does not mean that the same 
cause can have different results or different causes have 
the same result, any more in the mind than in physical 
nature. Different total statuses of the person never do lead 
to one absolutely identical status. It means simply that 
when certain accessories are neglected, one same conse- 
quence follows a hundred different stimuli. 

We can then analyze the stream or web or panorama of 
intellectual life, finding in it ideas. These occur not hap- 
hazard, but always by some cause. Chief among the causes 
are the bonds or links whereby one idea tends to be fol- 
lowed by a certain other idea. One man may have per day 
or per lifetime many more different ideas than another 
man. Of two men having the same number of ideas, one 
may have many more different connections than the other 
man has. For example, individuals A and B may each be 
capable of the ideas a, h, c, d, and the movement c. A may 
have only the connections a — > e, b —> e, c-* e, and d-*^ e. 
B may have' the connections a — > he, b -*■ ac, c — > dh, d — > ae, 
ab abcc, ac —> bbed, ad abce, be abed, bd aace, and 
others. 

This greater fund of ideas and connections is partly due 
to larger life and more varied and stimulating life, but it 
may be and certainly is partly due to original nature. It 
has some anatomical or physiological cause or parallel. Our 
hypothesis regards this anatomical cause or correspondent 
of the original possibility of having more such connections 
(call it C) as the cause of the original differences in intel- 
lect among men. It also supposes that the correspondence 
is such that C can be analyzed into a number of c’s so that 
a C which allows the formation of many connections with 
ideas has many c*8, whereas a C which allows the formation 
of only a few has few c’s. As we stated it, C consisted of 
c*s, one c meaning the possibility of the formation of one 
connection, two c’s meaning the possibility of forming two 
connections, and so on. This form of statement was chosen 
primarily for clearness and brevity. It needs amendment, 
or at least explanation. 
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Wliether there is any such one to one corespondence be- 
tween facts of anatomy and physiology, and facts of mental 
association, bonds or links in behavior is, however, not of 
much consequence to the hypothesis. What is essential to 
the hypothesis is that by original nature, men differ in re- 
spect of the number of connections or associations with 
ideas which they can form, so that despite identical outside 
environments, some of them would have many more than 
others. “The number of c’s a man has” means simply the 
original constitutional basis of the number of ideational 
connections which he has. It is highly probable that the 
original basis of quantity of connections is itself a matter 
of quantity, that a more potent C is one that has more of 
something than a less potent G has. But it is not necessary 
for the hypothesis that this should be so. So our hypoth- 
esis may better be amended to read : 

Let Ca, Ca4b, Ca+b+c, Ca+b+c+d, and so on, represent original 
natures such that with identical outside environments, the 
man having Ca will form a connections, the man having C.+b 
will form a + b connections, and so on. Then, with iden- 
tical outside environments, the amount of intellect wdiich a 
man manifests, and the extent to which he manifests 
“higher” povrers than other men, will depend largely upon 
his C/ 

Negatively, the hj’^pothesis asserts that no special quali- 
tative differences are required to account for differences in 
degree of intellect ; the higher processes or powers have no 
other basis in original nature than that which accounts for 
differences in the number of bonds of the associative type. 

The reader who is impatient with these subtleties may 
forget them all with no great loss. The gist of our doctrine 
is that, by original nature, the intellect capable of the high- 
est reasoning and adaptability differs from the intellect of 

^ Certain other inner conditions, su^ as the strength of curiosity, idie saUs- 
fyingnesB of thought for thought ja&e, and the appeal of non-intelleetual 
activities, in so far as they distinef from the man’s C, would have to be 
allowed for to make a perfect prediction. 
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an imbecile only in the capacity for having more connec- 
tions of the sort described. 

The bearing of the hypothesis upon the problem of mea- 
surement lies in the fact that we may be able^for many pur- 
poses to replace our measurement via a s'ampte inventory 
of tasks, hy a more or less direct measurement of^ C. If we 
can get access to C so as to measure it (and if the hypoth- 
esis is valid) we can measure intellectual capacity, and can 
measure it perhaps at a very early age. If also one C does 
vary from another simply by consisting of a larger number 
of c^s we have a single variable in the most convenient of 
all units. 

Any person familiar with the finer anatomy of the 
brain will at once think of the number of possible contacts 
(or possibly coalescences) of the fibrils of axones with den- 
dritic processes in the associative neurones which act in 
perception, thought, and speech as a highly probable C. 
We have had it in mind as the possible C which we should 
investigate first if opportunity offered. We do not, how- 
ever, make the hypothesis depend upon this particular C. 

EXPBKIMEA'TAL VEKIFICATIOST OF TUB QUANTITY HYPOTHESIS 

The hypotliesis may be submitted to an almost crucial 
test by determining the correlations within the upper half 
of intellect, those within the lower half, and those between 
the upper and the lower halves. Do the ^^higlier” abilities 
of selective and rilational thinking, abstraction, generali- 
zation and organization display close interdependence 
among themselves and marked independence of the 
^ ‘lower” or purely associative abilities? 

At our suggestion, Mr. J. W. Tilton has made this test 
in the case of 250 boys at the time of graduation from 
Grade 8. As measures of the “higher” or “control” abili- 
ties, he used sentence completions, arithmetical problems, 
and analogies tests. As measures of the more purely “as- 
sociative” abilities, he use^- vocabuisjy tests, routine and 
informational arithmetic, and information tests. 

The follovdng is a Sample of the arithmetic: 
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I. E, R. ARITHMETIC, ASSOC., II 


Add: 


a. 1>. 

c. 

d. 

e. 



/. 

g- 

f 


9f 3 

wk. 4 

da. 

7 

lb. 12 oz. 

1* 

i 1 

2 

5i 4 

wk. 2 

da. 

8 lb. 5 oz. 

2f 

i_ Jl 

5i 

6i 2 

wk. 6 

da. 

6 

lb. 14 oz. 

11 

Multiply : 








h. 

i. 

3’ 



Je. 

i. 


254 

9.6 

3 ft. 5 

in. 


16 

12% 


6 

4 

5 



, 2^, 

16 


Divide : 








m. 

n. 

o. 



P- 


50-5^7 = 

6 

1 .138 

3* 


= 

5 ^ E • 


Write the amswers 

to these questions: 






g. 1 cent is how 

many 

mills? 

- 

- 





n 1 pint is bow many gills f 

1 square mile is how many acres f ~ - 

t. How much is 20% of $601 _ ~ « 

u. How much is i3i% of $200? — - - ~ 

V- Which months have only 30 days? 

i£>. 1 rod is how many feet? - - 

a?. 1 acre is how many square rods? 
y. 1 meter is how many inches? 

0 . What is the square root of 64? 
oo. What is the cube root of 64? 
hh. 6* equals how many? 

The following is a sample of the information tests : 

I. E. R. INFORMATION, E 2 AND F 1 
Write or print >our name and age and grade in school here very plainly. 
Name Age Grade 

In each of 'the sentences below, you have a choice among four words. 
"Draw a line under the one of these four words which makes the truest sentence. 

E 2 

1. The Gnome engine is chiefly used in airplanes automobiles trac- 

tors motorcycles. 

2. Vinegar is made from picric acid apples bark lemons. 

3. Adobe is the name of a building material Indian tribe Chinese offl- 

cial flower. 

4. One of the books of the Bible is Jacob Jesse Joshua Judah. 

5. Oxo is the name of a meat extract automobile cigar toothpowder. 

6. Lille is in Belgium England France Switzerland. 

7. Queen Elizabeth of England was born about 1425 1525 1625 

1725. 


29 
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8, From Leningrad to Moscow is about 500 mi. 1,000 mi. 1,500 mi. 

2,000 mi. 

9. Corot is the name of a musician painter pigment general. 

30. The angle of incidence is equal to the angle of coincidence reflec- 

tion refraction subsidence. 


F 1 

1. The namber of rows of kernels on an average ear of corn is about 

5 15 25 35 

2. The ten commandments are called the decagon decalog decament 

decern vii ate. 

3. The ratio of the size of Africa to that of Europe is about 2 to 1 3 

to 1 G to 1 9 to 3. 

4> Brahmaputra is the name of a flower goddess language river. 

5. A pint can full of lead would weigh about 6 lb. 32 lb. 18 lb. 

24 lb. 

6. Hydrogen becomes liquid at about - 300° C. -150° C. -10° C. 

4 60° C. 

7. The number of a crab*s logs is four six eight ten. 

8- One inch equals about 2 cm. ill cm. cm. 22 cm. 

9. An irregular four sided figure is called a scoiium trapezium paral- 

lelogram pentagon. 

30. One of the books of the Bible is David Eleuzar Leviticus Uzziah. 

The correlations (corrected for attemiatioii), which are pre- 
sented in Table 131, f^five a clear answer. In this group the 
‘‘higher^^ abilities correlate as closely with the associative 
abilities as the hijijher do inter se, or as the associative abil- 
ities do inter se. The avera^^e of these six cases is .bbS 
.029 (l\ K.). The average of the three cases of “higher” 
with “higher” is .544. The average of the three eases of 
“associative” with “associative” is .571. The average of 
the six oases of “higher” with “associative” where the 
content differs just as much as it does in the “higher with 
higher” and in the “associative with associative,” is .577 ± 
.021 (P. K.). The three cases Co. with Voc., Ar. Coiit. with 
Arith. Ass., and Anal, with Inf. where the content is similar 
(words, numbers, and facts), have an average correlation 
of .71. 

If we apply the attenuation formula so as to measure 
the relation between (a) what is common to any two of the 
associative tests with words, numbers, and facts, and (b) 
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what is conmion to any two of the “higher^' tests with 
words, numbers, and facts, we obtain nine correlations® 
whose average is 1.07 dz .05 (c). If we eliminate the influ- 
ence of the correlations between higher and lower with simi- 
lar content (not using Tcov> rAnmt, or rAcAa) the correlation 
between what is common to the associative abilities and 
what is common to the higher abilities, has an average* of 
1.00 d= .03(a). 

There thus seems reason to refer the higher, originat- 
ing, directing abilities to much the same fundamental causes 
as the associative. The higher powers arc in their causa- 
tion as much like the lower as like one another. This is not 
because the correlations are insensitive indices. On the 
contrary, similarity in the content or data thought about, 
raises the correlation from .58 to .71. Nor is it because the 
subjects of the experiments did not have and use the higher 
abilities. 

We have extended, and, in general, confirmed, Mr. Til- 
ton’s findings by experiments of the same general nature. 

458 pupils in Grade 11 in city K were tested with 350 
vocabulary tasks and also with two forms of the I. K. R. 
Selective and Relational Thinking, Generalization and Or- 
ganization examination.’^ 

670 pupils in Grade 11 in city K, closely similar in abil- 
ity to the 458 just mentioned, were tested witli over a hun- 
dred reading tasks and also with the two forms of the I. K. 
R. Sel. Rel. Gen. Org. examination. 

The raw correlations of the general intelligence score 
with the total vocabulary score (sum of rights) and with 
the total reading score (sum of rights) were .72 and .73, re- 
sj)ectively. Q^he correlation of tlie general intelligence score 
with that from another similar pair of examinations is .92 
by the Spearman-Brown formula, the correlation of one 

B 1.01, 1.06, .82, 1.31, 1.26J, 1.07i, 1.11, l.OG, and .97^, with a median 
of 1.06. 

<• 1.00, 99i, ,77, 1.09, 1.09, 1.04, 1.01, 1.064, and .90, with a median of 1.01. 

^ This is a composite of stock tests of so-called general intelligence. 
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form with the other being .85. The correlation of the vocab- 
ulary score with that of another similar examination is .98 
by the Spearman-Brown formula, the correlation between 
two random halves thereof being .97. The correlation of 
the reading score with that of another similar examination 
is not known, but will not be far from .9. Using .92, .98, and 
.90 for these self-correlations, we have .76 and .80 as the 

TABLE 131. 

INTBHCOBREI4ATIONS (COBBECTED FOB ATTENUATION) OF SENTENCE COMPLETION 

(Co), Vocabulary (V), Arithmetic Control (Ac), Arithmetical 
Association (Aa), Analogies (An), and Information 
(Inf), in 250 Pupils op Grade 8^, (Compiled 

JI'IIA/JIT J. Ur* XXiyX’LTXy L P* 


Control with Control Control with Asso. Control with Asso. 

(content different) (content similar) 


Co An 

.522 Co Inf 

.722 

Co V 

.865 

Co Ac 

.523 Co Aa 

.550 

An Inf 

.604 

An Ac 

.587 An V 

.563 

Ac Aa 

.643 


An Aa 

.486 




Ac V 

.491 




Ac Inf 

.575 



Asso. with Ab 80 « 




V Inf. 

.787 




V Aa 

.433 




Inf. Aa 

.592 




Sdlf-CorrelationB 
V-I V-II 

.815 ± .0143 




Inf I Inf II 

.600 ± .0273 




Aa I Aa II 

.829 dz .0133 




Co I Co II 

.744 ±: .0190 




An I An II 

.920 ±: .0103 




Ac I Ac II 

.950 ± .0041 





coefficients corrected for attenuation between a stock intel- 
ligence score and vocabulary and reading, respectively. 
The mere knowledge of single words seems almost as ‘‘in- 
tellectual’* as the comprehension of paragraphs. 

If we use for each individual a level score representing 
the degree of difficulty at which he can succeed with 50% of 
the tasks, a similar result is obtained. The coefficients of 
correlation for the group in question are : 
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Corrected for 
Raw attenuation* 

General intelligence score with vocab- 
ulary .72 .77 

General intelligence score with compre- 
hension of paragraphs .66 .76 

We have measured 100 university students in (1) a com- 
posite of sentence completions and comprehension of para- 
graphs, (2) a composite of picture completions, pictorial 
analogies, and geometrical analogies, (3) arithmetical prob- 
lems,® (4) a vocabulary test, and (5) an extensive informa- 
tion test.^® The intercorrelations are shown in Table 132. 

The ‘^higher*’ abilities show an average corrected cor- 
relation among themselves of .48 by P and .35 by S; the 
‘‘lower” show a correlation of .67 by P and .59 by S. The 
average for a “higher” with a “lower” is .47 by P and .38 
by S. Ill general, the correlation is nearly as close between 
a “higher’ and a “lower” as within the higher or within 
the lower. 

The estimated correlation between what is common to C 
and A and what is common to V and Inf is .90. The esti- 
mated correlation between what is common to C and Pic 
and what is common to V and Inf is .78. The estimated 
correlation between what is common to A and Pic and 
what is common to V and Inf is . 56. The average is 
.75 ± .08(a). 

Additional evidence is found in the correlation in the 
case of 126 pupils in Grade 5^ for each of whom summation 

fi The self -correlation of the vocabulary level-score is *94. The self -corre- 
lation of the reading level-score is approximately .80 (.77 with a level-score 
from a less extensive test). 

» The arithmetical tasks were not hard enough to measure the ability of 
the group well, and the correlations would probably be considerably higher 
with an adequate set of mathematical tasks. But they would hardly surpass 
the information correlations. 

This was not as purely a test of associative thinking as would have 
been most desirable, a certain amount of organization and inference being 
of assistance in some of the tasks; but it was so to an enoimously greater 
extmt than the other composite. 



428 


THE MEASUSEMEHT OF IHTEIXIGEHOB 


H 

eo 

QO 

CO 

AO 

GQ 



CO 

C£> 

Ah 

oo 


eo 


iq 


iq 

CO 


.W 

h gOQ 

0 ^ 

s 

t .w 

I o CQ 
I PU 


00^0 
lO ^ 03 

CO CO 

03 CO 


^ < 
^ 1-3 

C I 
« I 

SB 

^ SB 
r o 
W g 

3 4 

W X « 

l-q ^ II 

« oi cw 

-aj M ^ 


g-'H® 


o »rt 05 
CO C3 >0 


;?! 

,-H 03 CO l-^ 
lO ^ ;0 

• • • • 


s .w 

.S §“ 

C« ^ A 

'S Ah 


03 <0 
AO OJ rH 


OO CO o> 
AO 03 03 


^ . M 

Is" 


Gd 

f§ -3 

d 'C .S o 
U <J Pa (> 


rH CO 04 ■<«' 



THE STATURE OF INTEIJUBOT 


429 


scores in C, A, V, D, and Inf were available. The reliabili- 
ties of these scores have not been determined at all exactly, 
bnt they are high and approximately equal, each represent- 
ing about 40 minutes of work. Table 133 presents these 
correlations. The average of the intercorrelations of C, A, 
and D is .66. The correlation between V and Inf is .81. 
The average of the intercorrclations of C or A or D with 
V or Inf is .62. The change from “higher” to “lower” 
abilities does not reduce the correlations as much as the 
change from words to numbers within the higher abilities. 
C and D correlate .81, whereas C and A correlate only .64 ; 
and D and A only .52. 


TABLE 133. 

The Int£ECOBkex#atioks op Theee Tests op the Higher and Two Tests op 
THE Lower or Associative Thinkino. 126 Pupils in Grade 5^. 

Tub CORRBIiATIONS ARE ALL RaW CORRELATIONS 
BY THE Sheppard Formula. 



A 

V 

D 

Inf. 

c 

.64 

.75 

.81 

.59 

A 


.52 

.52 

.41 

V 




.81 

D 




.64 


The correlation between what is common to C and A 
and what is common to V and Inf is given by 


V.75 X .59 X .52 X .41 
V-64 X .81 


or .77. 


The correlation between what is common to D and A 
and what is common to V and Inf is given by 


V.8Q X .64 X .52 X .41 
V-52X.8i 


or .71. 


The correlation between what is common to C and D 
and what is common to A and D is given by 
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V-64 X .81 X .52 X 1.00 
^1 X .52 


or 1.11. 


The average is .86 ± .10 (error is in terms of a t-o). 


SUMMARY 

We may combine the results from Mr. Tilton ^s 250 cases 
and our two groups of 100 and 126 cases roughly, giving 
equal weight to the last two determinations and as much 
weight to Mr. Tilton’s determination as to the other two 
together.'^ The weighted average correlations are : 


Higher with higher .53 

Lower with lower .64^ 

Higher with lower ■ ^ .57 

What is common to two higher vnth 

what is common to two lower .94 


These facts are almost crucial. They prove that mere 
association and the higher abilities have in the main the 
same cause. Almost all of whatever is common to the one 
sort is common to the other sort. If we are to avoid the 
conclusion that associative ability is this cause, we must 
either place the causation of associative ability in the higher 
ability, or seek a common cause for both which is different 
from either, such as a general mental energy or vitality. 
The first of these assumptions is absurd, because associa- 
tive ability occurs abundantly without any trace of the 
higher abilities, but these never occur without it. In the 
lower animals, in idiots, and low imbeciles, and in the young 
infant, mental connections are formed without the api)ear- 
ance or use of abstraction, generalization, or relational 
thinldng. If either is to be derived from the other, it is 
surely best to derive the higher abilities from the associa- 
tive abilities. The second assumption is tolerable, though it 

His determination probably deserves even more relative weight than 
this because his test material was better adapted to bring out any differences 
between the higher and the lower forms of intellect. If more weight is given, 
the higher and lower become still less distinguishable. 
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seems defensive and evasive. It also is entirely empty and 
meaningless until the “energy” or “vitality” is expressed 
as some fact known to science. What shall that fact bet 
Until that fact is chosen, the doctrine that ability in con- 
nection-forming, and ability in the higher processes, have a 
common cause which is not the former, is a mere statement 
of ignorance. We can think of no fact so suitable as C, the 
physiological parallel of number of mental connections. 
The cause must not be a too general vigor or health or en- 
ergy or sensitivity or conductivity of neurones. For the 
correlations between intellect and other functions of the 
nervous system are very far from perfect. Between intel- 
lect and mental health or balance, between intellect and sen- 
sori-motor skill, between intellect and sensory acuity, be- 
tween intellect and morality — no one of these correlations 
would be as close as the correlation between the associative 
ability and “higher” ability within intellect. 

We do not maintain that C is the sole cause of intellect 
in original nature, so that two persons with identical num- 
bers of C ’s and identical training will necessarily have iden- 
tical intellectual achievement. We have already noted, as 
factors which play a part, strength of curiosity, satisfying- 
ness of thought for thought’s sake, and competition from 
non-intellectual activities and interests. Other thing must 
be equal, such as health and energy. There is also perhaps 
a capacity for having the neurones act with reference one to 
another, that is, with integration, whose low or negative 
extreme is pronounced dissociation as in hysteria, and 
whose high or positive extreme appears as a notable good- 
sense or adequacy in the use of one’s experiences. This 
capacity may be largely irrespective of C. There is also 
perhaps a capacity for resisting intellectual panic and con- 
fusion, whose low or negative extreme is mania or “flight 
of ideas” and whose high extreme is a notable steadiness 
and regulation of each individual connection by the general 
set or adjustment of the mind at the time. The strength of 
this capacity may be largely irrespective of C. 
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These and other possible qualifications do not impair the 
value of the hypothesis. They amount to making distinc- 
tions between basal intellectual capacity and interferences 
with it or handicaps to it by disassociation as in hysterical 
lapses, or by irrelevance as in mania. We should have just 
as much need to make these distinctions if we attributed 
differences in intellectual capacity to differences in the 
quality, or in the shape, or in the chemical action of neu- 
rones. Nor would two sorts of neural action qualitatively 
different relieve us from them. 

We shall not discuss general arguments pro and con in 
this report, but will simply note that both the phylogeny 
and the ontogeny of intellect seems to us to show selection, 
analysis, abstraction, generalization, and reasoning coming 
as a direct consequence of increase in the number of connec- 
tions ; and that what little is known of the status of the neu- 
rones in very dull individuals is in harmony with the quan- 
titative theory.^® 

12 See especially Hammarberg, Studier ofver Idiotlcns, Klinik och Patologi. 



CHAPTEE XVI 


The Measurement of Original. Intellectual. Capacity 

AND OF AcQUIRM) INTELLECTUAL ABILITY 
THE PRESENT STATUS OF OPINION 

Psychologists are often credited with the opinion that 
the intelligence examinations which they have devised, such 
as the Binet or Army Alpha or Army Beta, measure an in- 
dividuaPs original intellectual capacity, irrespective of the 
opportunities which he has had, or the time he has spent in 
intellectual activities, or the zeal with which he has engaged 
in them. 

So extreme a view is not, however, held hy any of the 
leaders in this field. The nearest approach to it that we 
have noted in their statements is that of L. S. Holling- 
worth, who defines intelligence as “the capacity for learn- 
ing, the capacity for comprehending and making adapta- 
tions to the environment** which “cannot be acquired by 
any course of training** [*23, p. 192] ; and says of the Stan- 
ford Binet “It measures intelligence** [*23, p. 67], when 
discussing means “of singling out intelligence from all the 
other factors which complicate efficiency in school work** 
[*23, p. 62]. 

The following quotations from Colvin, Whipple and 
Terman are representative: 

“There is no reasonable doubt that the present intelli- 
gence tests do indicate to a fair degree native ability to 
learn** [Colvin, *23, p. 336]. 

Colvin and MacPhail (speaking of Professor Bagley*s 
article, “Educational Determinism or Democracy and the 
I. Q.”) say: “He is right if he means that it is not always 
easy to determine what this innate learning capacity, thi 
native intelligence is, and that mistakes in individual caue 
may be made ; but he is wrong if he would convey th/f the 

433 dectual 
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that in general it is not possible to determine within reason- 
able limits what the native learning capacity of the child is, 
provided adequate intelligence tests are employed and vari- 
ous common sense precautions* taken [^22a, p. 114]. 

Terman writes: a matter of fact, all the * intelli- 

gence testers’ will readily agree with Mr. Lippmann that 
their tests do not measure simon pure intelligence, but al- 
ways native ability, plus other things, with no final verdict 
yet as to exactly how much the other things aifect the 
score. However, nearly all the psychologists believe that 
native ability counts very heavily” [’22, p. 119] ; and else- 
where, ^‘It would, of course, be going too far to deny all 
possibility of environmental conditions affecting the result 
of an intelligence test. Certainly no one would expect that 
a child reared in a cage and denied all intercourse with 
other human beings could by any system of mental mea- 
surement test up to the level of normal children. There is, 
however, no reason to believe that ordinary differences in 
social environment (apart from heredity), differences such 
as those obtaining among unselected children attending ap- 
proximately the same general type of school in a civilized 
community, affect to any great extent the validity of the 
scale” [’16, p. 116]. 

Whipple writes: ‘‘In presenting these results, it ought 
to be made clear at the outset that no psychologist is foolish 
enough to suppose that native intelligence is the sole factor 
in academic success; all that is contended is that it is one 
factor, and probably the most important single factor, and 
that it is measurable by wholesale rapid methods with a 
reasonable degree of precision” [’22, p. 262] ; and else- 
where, “We know that the organism arrives at approximate 
maturity of growth in stature and in many other physical 
traits in early adolescence; the fact that our test scores 
'ndicate the maturing at about the same time of whatever 
is we are measuring, like the fact that, regardless of 
ological age, the correlation between stature and men- 
is high, may very well indicate that our tests are 



OBIGIlirAli AND AGQUIBED INTBIAjECTUAL ABILITY 435 


also measDring an intrinsic capacity which matures accord- 
ing to laws of its own and with relatively little influence 
from the environment’* [^23, p. 597]. 

On the other hand, there are some emphatic assertions 
that a person’s score in a stock intelligence examination is, 
in very large measure, the product of purely environmental 
forces. So Cyril Burt writes of the Binet: Errand boys 
and paper boys will answer smartly in the money tests. . . . 
The busy little housewife from an illiterate home, who there 
carries out the most intricate duties, will yet be unable to 
put those duties into words. The solitary child of a cul- 
tured family — ^profiting, perhaps, rather by daily inter- 
course with educated adults than by special inborn gifts — 
will respond with an information and a phraseology beyond 
anything he would -spontaneously invent or acquire. . . . 

‘‘Of these numerous intervening agencies the most 
potent is, without doubt, educational opportunity. Many 
of the tests — some of them withdrawn by Binet in his final 
revision — are sheer tests of school attainments. Beading, 
writing, dictation are learnt in English lessons; counting 
and addition and subtraction of money, in arithmetic les- 
sons; drawing from copy and drawing from memory, in 
drawing lessons ; the date is put at the head of every written 
exercise on every day of the term, and with equal regularity 
is never heard and never recollected on any day of the vaca- 
tion. Estimated by the Binet-Simon scale, therefore, a 
child’s apparent intelligence must depend in no small mea- 
sure upon his class in school” [’21, p. 175]. 

GENERAL PRINCIPLES 

We have hitherto defined and measured intellect without 
restrictions as to its origin, and without distinction between 
sheer ability at thinking and a love of thinking which makes 
one think oftener, longer, and harder. If, however, either 
the altitude or the area of Intellect CAVD is entirely due 
to an original capacity that is entirely independent of the 
kind or amoimt of training received and of the intellectual 
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interests inherited or acquired, the fact should be known. 
If analytic measures are possible whereby certain tasks or 
symptoms isolate and measure the original capacity, cer- 
tain others measure the element of interest or zeal, and still 
others measure the potency of the environments provided, 
such would be useful in many ways. It might be impossible 
to measure capacity separately from interest, and still be 
possible to measure inherited intellectual promise (due to a 
mixture of capacity and interest) separately from environ- 
mental alterations thereof. Or, it might be possible to mea- 
sure the sheer capacity apart from the interest factor, but 
not to sei)arate nature’s share from nurture’s. Either of 
these last two possibilities would be useful. 

We may best begin by certain simple axioms, or, more 
modestly, truisms. (1) If two men had been subjected to 
identical circumstances in life, each and every difference 
between them would be due to original nature ; if two were 
alike originally, all their later differences would be due to 
the circumstances of life. 

(2) In proportion as an intellectual task is one in re- 
spect of which all persons have had equally adequate train- 
ing, so that no conceivable classification by environmental 
opportunities would correlate at all with success in the task, 
that task is a measure of original capacity (plus original 
interest). 

(3) In proportion as a series of intellectual tasks gives 
on the whole as much advantage to any one set of environ- 
mental oiiportunitics as to any other set, that series is a 
measure of original capacity (plus original interest). 

(4) Intellectual tasks, success in which requires zero 
training and is uninfluenced by any kind or amount of train- 
ing, do not exist and cannot exist, at least not in shape to 
measure appreciable amounts of intellect. 

The first three axioms are self-evident and undisputed, 
but the fourth may seem to run counter to the beliefs, or at 
least the hopes, of some psychologists. Indeed, one is 
tempted to think that children who are set tasks in fillin g 
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form-boards or tracing mazes, never before having seen 
one, start at zero and get all the relevant training in the 
course of the test itself. But that is never the case above 
the lowest levels. The tracing of a maze rests upon habits 
of response to location and direction which we once 
learned; the filling of the simplest form-board depends 
upon habits of perception of shape and size which we learn 
as truly as we learn Euclid or shorthand. It is because we 
have all learned them and learned them early that we tend 
to forget that they are influenced by training. Because 
almost everybody has learned them, these tasks are, by 
axiom 2, more suitable (other things being equal) to mea- 
sure original capacity than shorthand or Euclid would be. 
But they cannot be said to require zero training. 

It is conceivable that by some direct method of examin- 
ing the finer anatomy or physiology of the neurones, as by 
some technique analogous to the X-ray technique, original 
intellectual power and interest may be separated from ac- 
quired ability without any relianc<3 upon axioms 2 and 3. 
But so long as we measure intellect by the production of 
intellectual products — success with intellectual tasks — ^we 
can never reduce environmental forces to zero; we must 
always seek to equalize them. 

THE USE OF NOVEL. TASKS 

One common method of obtaining som^ degree of equali- 
zation is to make the tasks novel, so that at least no person 
will have been taught to do that particular task by environ- 
mental forces. This has played a part in the disarranged 
sentence test, the number-series completions, and the mixed 
relations or analogies of Army Alpha, in the maze, cube, 
rhythm, picture-completion, and card-cutting tests of Army 
Beta, in a majority of the pencil and paper tests for young 
children unable to read, and in many others. Each year 
brings forth new selections or adaptations or inventions of 
tasks where intellect operates with novel data or with old 
data in new ways. 
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This method had the further alleged advantage that 
novelties were supposed to measure ability to analyze and 
infer and reason — ^the higher mental processes — ^better than 
more customary tasks could. We have seen that this ad- 
vantage may be of less moment and magnitude than has 
been supposed, since the “higher’^ and the “ lower proc- 
esses measure nearly the same abilities at bottom. 

The equalization of environmental influence obtained by 
novelty in and of itself has one notable practical disadvan- 
tage. Special coaching for the tests is likely to produce 
very great inequalities in favor of those who receive it. 
For example, the syllogism test of Rogers shown below will 
be made very muelT’ easier for many persons if they are 
taught to make a diagram representing the given facts by 
position along a line, as shown below. 


Fill ixL with conclusions which can be correctly drawn from the given facts 
in each set as shown in the hrst line. 


Giv6n Facts ConcItMions 

f therefore Y is thicker - than 

Z is thicker than X 
(1) H is as thick as Z 

V is thicker than H 

V is thinner than Y 

I therefore Z is 


therefore X is 
therefore Y is 
therefore X is 


V 
H 
H 

V 

V 




therefore 

B ia 


A 


D is greater than B 

therefore 

D is 


F 


B is equal to E 



(2) 

E is greater than F 

therefore 

E is 


A 


C is less than F 






A is greater than D 

therefore 

B is _ . 


C 



therefore 

A is 


- . P 


X 

Z 





thin 

H 

V 

Y 

thick 

less 

C P 

£ 

£ 

D 

A_ 

great 
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Tl££ USE ^fiLIAE TASKS 

A contrasting method is <> tasks that are so fanuliar 
that everybody has had somev^early adequate environ- 
mental stimulation to master ti»any person’s success 
depending consequently chiefly on intellectual cai»city 
and interest. Thus, “Are you a br a girl?” 
morning, afternoon, or evening?” hat month is it? 
“Name all the months of the year,” tya bow knot, nam- 
ing the days of the week, naming six c^ and other e e- 
ments of the Stanford Binet are tasks wh''bviously lack 
novelty. Their merit is not that trmg has little efEec 
on them, but that training treatil people somewhere 
nearly alike in respect of them. 

The equalization of the ej-onment’s influence by 
choosing tasks where it is aj^^te for nearly all, is aided 
by relying on the third than the second axiom and 
using an extensive s^_^^g things which the world m 
general stimulates p fL everybody to learn or do. Thus 
it may be argued thf', ^^^^1 number of such words known 
will be a better inc, . original intellectual capacity an 
interest than the " of any particular score o 

words. The envir ^nav decide that A learns word 

about things and j^^chanisms, that B learns words abou 
animals and planl , C learns words about peo^e 

and their actions ’ , ^.ell have less power over the 

total niimber leai , 

One method ff^e^tnating the original *“tors s^ks 

to equalize envir ^ influence in respect of the data 

or content invoh^J . w familiarity; and seeks t 

„ j - ed in the tasK, vy a** r.Ti#^ration 


reduce enviromr^ •;;j„en<» in respect of the 
with tte d8la..» the task, through ^ 

compretrt’'; or-ncrranhs whose words singly are au 

well b^xiension o p , . ■ ^f sentences about familiar 

h J^«on of unconventional 


facte the solution of unconventional 

ur OS in fanuliar word , . , ^og^g For example, as- 

,ithmetical problems' are typical cases. 


sei 


Z 7 Sltiplied by Bome number equals 63, ^bat is the numberl- 
30 
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smne that a thousand individuals aged 16 have been in at- 
tendance upon school in f^he United States eight years or 
more (of 150 school da'^^S or more) and are measured by a 
battery of tasks chosen with care (say, from those printed 
in this volume) to contain only words from the 4,000 known 
to almost all sucl^ individuals, and to require only such 
arithmetical facts/ and techniques as are taught in grades 
three and four. 

Differences an the degree of success Avith such a battery 
of tasks might rtiasonably be regarded as largely* indepen- 
dent of differences in .^chool environment save in so far as 
these differences themseilves were caused by original differ- 
ences in capacity and zeal^ 

THE USE OF A SERIES GBAiUED FOR SUSCEPTIBILITY TO 

f 

ENVLBOHMEHTAL IlyTPLUENCES 

We can then, by one or another ofVthese methods, select 
or devise a battery of intellectual tasKS or tests the score 
in which (in a group of individuals of Ahe same time, coun- 
try, and general manner of life) bids fajir to be determined 

B. What part of 1 6 equals half of 24 f 

C. How many quarters of a quarter equal half of Ja half? 

D. In the lines below, each number is gotten in ia certain way from the 
numbers coming before it. Study out what this way j/s in each line, and then 
write in the space left for it the number that should ctJ nie next. The first two 
lines are already filled in as they should be: 

1 2 4 6 8 10 12 

Samples | ^ jg 18 

1. 38 34 30 26 

2. 103 95 87 79 

3. 1 10 100 1000 

E. Write the numbers and signs in each line below in ' the proper order, sc 
that they make a true equation as shown in the two sample liner 'v* 

Sample lines } J ® 8 20 = + x 

1. 2236 15 =--x 

2. 2 6 6 7 10 = 1- + - 

3. 1 4 8 15 20 = + - - 

2 How largely will depend upon the diseiplinary values of school training 
and upon the extent to which our novelties axe really novel. 
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to a considerable extent by original nature, another battery 
the score in which a larger fraction of the score is due to 
circumstances, and so on. The application of this series of 
examinations graded from “least subject to environment '' 
to “most subject to environment’* would provide a partial 
analysis of a man’s intellectual ability and attribution 
thereof to his original nature and to his acquisitions. Thus, 
suppose that the examinations A, B, C, D, and E, when ap- 
plied to native-born white citizens of the United States, 
aged 21, depend respectively on nature and nurture in these 


proportions. 


Nature 


Nurture 



A 

7 


3 



B 

6 


4 



C 

5 


5 



D 

4 


6 



E 

3 


7 


Suppose that individuals I and II score as follows : 

A 

B 

C 

D 

E 

Total 

I 90 

80 

70 

60 

50 

350 

II 50 

60 

70 

80 

90 

350 


Then obviously, I and II, who are of equal present status, 
have it from very different causations. 1 had much the 
better original equipment, but has not much improved it. 
From such measurements, we could infer the relative con- 
ditions of individuals at the limit where original nature was 
10 and environment 0 in the causation. Thus in the case 
above, suppose all units to be truly equal and referable to 
a true zero point. Then by original nature, I would have 
120, or six times the original possibility of 20 that II would 
have. If the environment could have been made as favor- 
able for him as it was for II, he would have had a total 
score six times as large as II ’s. 

The importance of such analytic measures by a graded 
series, and of such inferences about the conditions at its 
limits, depends on (1) the surety with which we can pick 
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tasks to be differently sensitive to original capacity, (2) 
the extent to which the differential series will extend up 
and down from the modal condition of sharing of original 
nature and environment, whatever that may be, (3) the 
nearness of that mode to the condition where one hundred 
percent is caused by nature and zero percent by environ- 
ment, and (4) the restrictions which we have to impose on 
environmental differences in order to work the plan at all. 

Little has been known with surety concerning any of 
these matters ; and we have not had the time or facilities to 
make more than a beginning at the investigations which are 
needed. All our work concerns these problems in the case 
where environmental differences are limited to such as hold 
for white individuals born and bred in the United States, 
belonging to the same generation (born say, not over 20 
years apart), provided with opportunity to go to school for 
at least 6 years (or 900 school days) unless they were de- 
monstrably so stupid as to be unable to learn at school, 
and not deaf, dumb, blind, or insane. If, for example, we 
state that environmental differences cause only K percent 
of the differences found in a test in completing sentences or 
solving arithmetical originals, we do not mean that the per- 
cent would not exceed K in a group composed half of pres- 
ent-day Americans and half of their ancestors fifty thou- 
sand years ago ; or in a group composed half of present-day 
Americans and half of African pigmies.® 

In accord with the principles already stated, the tasks 
which would be chosen as especially indicative of original 
capacity are sentence-completions, arithmetical problems, 
and comprehension of paragraphs, especially such as re- 
quired the use of familiar data in new ways. At the other 
extreme would be the knowledge of single words and iso- 
lated informational items. 

"When we apply these two extremes, we find that they 
are really very close together. Either they do not measure 

A An investigation of tests which may be freed from these limitations in 
whole or in part is being made by a group of psyehologistsi with funds sup- 
plied by the Spelman Memorial. 
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two very different things, one the intellect a -man has by 
nature, and the other the intellect which he has acquired by 
training, or these two are in almost perfect correspondence. 
For the correlation between whatever is common to the 
tasks of one extreme and^whatever is common to the tasks 
of the other extreme is almost perfect. The evidence is the 
same as that brought forward by Mr. Tilton and by us in 
Chapter XV to prove that associative thinking and analytic, 
inferential, original thinking have almost identical roots. 

As a cheek, we may contrast, within each form of test, 
certain elements which home and school advantages would 
benefit less with certain other elements which they would 
benefit more. Thus, within the field of sentence-completion, 
we may divide our elements 65 to 130 into two halves 
as shown below, as a result of a consensus of expert opinion. 

Benefited More by Home and School Advantages 


66. The of the World were 

kinder than the kings and nobles 

Old. 

68. The of five and ten is fifteen. 

70. At time was progress rapid 

during the last half of the nineteenth 

74. The source wealth in Den- 

mark agriculture. 

75. The laws inheritance are for the most 

unknown. 

76. In to maintain health, one 


should have nourishing 

82. At ancient banquets the of the day seems 

have the chief 

of conversation. 

84. The Declaration affirms that 

the Creator all men with certain inalien- 

able — 

85. This was done a view caus- 

ing the of carfares 

three cents. 

93. One of the most difficult problems of representative 

is that of getting large assemblies to 

the work of legislation 

and efficiently. 
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98 . 

99 . 

100 . 

101 . 

102 . 

104 . 

106 . 

116 . 

119 . 

123 . 

124 . 

125 . 

127 . 

128 . 
129 . 


The word virtue is derived from a 

strength. 

The struggle for among the lower 

has a commonplace of modern scientific 

thought. 

The .... World wished to the 

New, 

Saddles and bridles no unimportant 

in the art. 

India is rich in of scenery and climate, 

the mountains to vast 

deltas raised a few 

above sea 

Undue consciousness often the fiow of expres- 
sion diffuseness is detrimental to a clear 

and exposition of our ideas. 

Throughout the river plains of northern India, two harvests, 

and, some proAunces, are 

each 

Few historians would the fact that Marx had 

a larger and more thorough on the social 

of his time any 

living man. 

Let us very briefly examine the social forces 

at work concentrating or 

the ownership wealth. 

Modern inherits 

innate pugnacity and all the love of glory of 


Let the class that itself to transportation, for 

example, working and the disastrous 

to the rest of the can 

scarcely imagined. 

The monuments of PersepoHs 

the use incense a s 


in ancient Persia as Baby- 


lonia. 

The orderly peaceful of our 

industrial mechanism is a of public 

, and be secured in 

way or 

Ever since the hearing before him the gov- 
ernor giving 

spare moment a of the case. 

So far the displeasure of 

the people by the will of their represen- 
tatives, a President generally gains by the 

bold use of his veto power. It conveys the 
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firmness ; it shows 


has a view and does 

to give effect to it. 


Benefited Less by Home and School Advantages 

65. It may effort and a long but 

the result is sure. 

71. He will come to the meeting 

the fact he 

rather stay quietly at home. 

73. No what happens wrong is 

right. 

77. His friends, — wished to dissuade him from this 

undertaking, asserted that he followed 

their advice would withdraw their sup- 

port. 

78. It would several pages to 

contain the list. 

79. Standing beside the grave 

great Englishman enough 

for us to know lived and 

died, and made the his heirs. 

80. You may safely conclude that you in yourself 

the means of at the truth. 

81 the fact that you disagree with me, I shall 


83. As 


continue to aid you. 


86. The 


ably it existed 


the treasure he had come to seek, prob- 
in his own 


of a man is to be useful to his 


87. They who are miserable have medicine other 

hope. 

88. The best is too 

him. 

89. The sublime the ridiculous 


often so nearly 


it is 


to class them separately. One step above 

the makes the ridiculous, and one step 

the makes the 

again. 

90. spite many severe 

he is still alive the of ninety- 

one. 

91. It appears whether his debts will be paid. 

94. If the of the year were holi- 
days playing be tedious 

working. 
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95. Gratitude 

sheds 

heart. 

97. Impatience, while it 


the fairest 


that 


perfume 


us to surpass 

generations, disposes to 

overrate their happiness. 

103. The American boy wishes go 

college and to go, has 

only his own weaknesses to thank for it. 

104a injury nor retaliation nor warding off 

by evil is ever 

105. Knighthood and Chivalry are words 

are nearly not 


synonymous. 

107. a man time sufficient for all 

laudable pursuits, and sufficient for all 

generous purposes, he is free 

shadow of blame or reproach. 

108. Maize contains small a proportion of nutri- 

tious matter it not 

for horses which fast 

work is 

109. The drafting a measure depends 

the pains and skill exerted by its 


110 . 


115. 

117. 


is natural that being dissatisfied with the 

, we should form a too 

estimate of the past. 

Virtue knowledge, and is the 

fruit of ignorance. 

He will do as you request 

his own feeling. 


117a. Where in nature is grandeur 

displayed as in the Grand Canyon? 

126. He would assign no reason his action 

to his 


to 


enemies. 


Within the field of arithmetical problems the same con- 
sensus lists ten problems as those benefited most by home 
and school advantages, and nine as benefited least by them. 

We put these “Most” and “Least” tasks into four 
groups: Ma, Mb, La, and Lb, Ma and Mb being random 
halves of the M’s, while La and Lb are random halves of 
the L’s. 
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In a group of two hundred normal-school students, the 
correlations are as follows : 

Ma with Mb, .74 

La with Lb, .81 

Ma with La, .69 

Ma with Lb, .66 

Mb with La, .67 

Mb with Lb, .72 

The correlation between the ‘‘Most’^ and the “Least*' 
is thus nearly as high as the correlation within the “Most" 
or within the Least," and the correlation between what is 
common to the “Most" and what is common to the “Least" 
is .88. 

On the whole, the difficulties in the way of analyzing a 
man's intellect into the contribution of nature and that of 
nurture by the use of tasks much subject to environmental 
influences, and tasks little subject to them, are very great. 
The method is sound, but hardly practicable. 

THE TEST AND EBSUL.TS OF BURT 

Burt has sought to measure the relative shares of intel- 
ligence, school environment, and age. His work is so im- 
portant that we quote the report of it in full. In estimating 
the meaning of his results, we must bear in mind that his 
“intelligence" is in reality the score in the test (Test 29. — 
Graded Reasoning Test) quoted below. His final conclu- 
sion, that B (the Binet Mental Age) = .54S + .331 .llA, 
should then be modified to read as follows : 

“Of the gross result, then, one-ninth is attributable to 
age alone, one-third to the ability measured by the Burt 
Reasoning Tests alone, and over one-half to the ability mea- 
sured by school attainment alone." 

Test 29. — Graded Beasonino Tests 
(Short List) 

Tom runs faster than Jim : 

Jack runs slower than Jim. 

Which is the slowest of the three? 


1 . 
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7 Years. 

2. Kate is cleverer than May. 

May is cleverer than Jane. 

Who is the cleverest — Jane, Kate, or Mayf 

3. I have bought the following Christmas presents; a pipe, a 

blouse, some music, a box of cigarettes, a bracelet, a toy 
engine, a bat, a book, a doll, a walking-stick, and an 
umbrella. 

My brother is eighteen: he does not smoke, nor play cricket, 
nor play the piano. 

I want to give the walking-stick to my father, and the umbrella 
to my mother. 

Which of the above shall I give my brother ? 

8 Years. 

4. I don’t like sea voyages: 

And I don’t like the seaside. 

I must spend Easter either in Prance, or among the 
Scottish Hills, or on the South Coast. 

Which shall it be? 

5. The person who stole Brown’s purse was neither dark, nor tall, 

nor clean-shaven. 

The only persons in the room at the time were — 

1. J ones, who is short, dark, and clean-shaven : 

2. Smith, who is fair, short, and bearded : 

3. Grant, who is dark, tall, but not clean-shaven. 

Who stole Brown's purse? 

9 Years. 

6. Three boys are sitting in a row: 

Harry is to the left of Willie : 

George is to the left of Harry. 

Which boy is in the middle? 

7. In cold, damp climates, root crops, like potatoes and turnips, 

grow best : 

In temperate climates, there are abundant pastures, and oats 
and barley flourish : 

In sub-tropical climates, wheat, olives, and vines flourish: 

In tropical climates, date-palms and rice flourish. 

The ancient Greeks lived largely on bread, with oil instead 
of butter : they had wine to drink and raisins for fruit. 
Which climate do you think they had? 
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10 Years. 

8. There are four roads here : 

I have come from the south and want to go to Melton. 

The road to the right leads somewhere else : 

Straight ahead it leads only to a farm. 

In which direction is Melton — ^North, South, East or West! 

9. The doctor thinks Violet has caught some illness. 

If she has a rash, it is probably chicken-pox, measles, or 
scarlet fever; 

If she has been ailing with a cold or cough, she may develop 
whooping-cough, measles, or mumps. 

She has been sneezing and coughing for some days : and now 
spots are appearing on her face and arms. 

What do you think is the matter with Violet ? 

11 Years. 

10. Where the climate is hot, gum-trees and rubber will grow : 
Heather and grass will grow only where it is cold ; 

Heather and rubber require plenty of moisture: 

Grass and gum-trees will grow only in fairly dry regions : 
Near the river Amazon it is very hot and very damp. 

Which of the above grows there? 

11. Father has just come home in a brand new overcoat ; there is 

clay on his boots and flour on his hat. 

The only places he can have been to are Northgate, Southgate, 
Westgate, or the City; and he has not had time to go to 
more than one of these. 

There is no clay anywhere in the streets except where the 
pavement is up for repair. 

There are tailors' shops only in Southgate, Westgate, and the 
City. 

There are flour mills only in Northgate, Westgate, and the City. 
I know the roads are not being repaired in the City, though 
they may be in the other places. 

Where has father been? 

12 Years. 

12. Field-mice devour the honey stored by the humble-bees; the 

honey which they store is the chief food of the humble-bees. 
Near towns, there are far more cats than in the open country. 
Cats kill all kinds of mice. 

Wliere, then, do you think there are most humble-bees — 
in the neighbourhood of towns or in the open country? 
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13. I started from the church and walked 100 yards : 

I turned to the right and walked 50 yards : 

I turned to the right again and walked 100 yards. 

How far am I from the church? 

13 Years. 

14. A pound of meat should roast for half an hour: 

Two pounds of meat should roast for three-quarters of an hour ; 
Three pounds of meat should roast for one hour: 

Eight pounds of meat should roast for two hours and a quarter : 
Nine pounds of meat should roast for two hours and a half. 
From this can you discover a simple rule by which you 
can tell from the weight of a joint for how long it 
should roast? 

15. What conclusion can you draw from the following facts? 
Iron nails will not float in a pool : 

A cup of pure gold dust weighs nearly twenty times as much 
a.s a cup of water of the same size : 

If you drop a silver sixpence or a copper coin into a puddle, 
it will sink to the bottom : 

A cubic inch (about a tablespoon ful) of water weighs less than 
half an ounce; a cubic inch of brass weighs over two 
ounces : 

A leaden weight will drop to the bottom of the ocean. 

Sum up all these observations in one short statement of the 
following form: “Most are 

7 7 


14 Years. 

16. John said: “I heard my clock strike yesterday, ten minutes 

before the first gun fired. I did not count the strokes, but 
I am sure it struck more than once, and I think it struck 
an odd number.” 

John was out all the morning from the earliest hours: and his 
clock stopped at five to five the same afternoon. 

When do you think the first gun fired? 

17. Captain Watts and his son James have been found shot — ^the 

father in the chest and the son in the back. Both clearly 
died instantaneously. 

A gun fired close to the person — as, for example, when a man 
shoots himself — will blacken and even burn the skin or 
clothes ; fired from a greater distance, it will leave no such 
mark. 
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TJiC'two bodies were found near the middle of a large hall used 
as a rifle range. Its floor is covered with damp sand, 
which shows every footprint distinctly. Inside the room 
there are two pairs of footprints only. A third man stand- 
ing just outside the door or window could aim at any part 
of the room ; but the pavement outside would show no foot- 
marks. 

Under Captain Watts’ body was found a gun: no such weapon 
was found near James. 

In each case the coat, where the bullet entered, was blackened 
with gunpowder, and the cloth a little singed. 

Captain Watts was devoted to his son, and would have died 
sooner than harm him purposely : hence it is impossible to 
suppose that he killed him deliberately, even in self- 
defence. But some think that James secretly disliked his 
father, and hoped to inherit his fortune at his death. 

(1) Was Captain Watts’ death due to murder, accident, or 
suicide ? 

(2) Was James’ death due to murder, accident, or suicide? 

[Burt, '21, pp. 239-242.] 

“For every child in an entire school, comprising just over three 
hundred pupils aged between seven and fourteen, I have secured 
the following measurements: first, the child’s age; next, his school 
attainments, measured by an educational examination, the results 
being revised by the teachers ; thirdly, his intelligence measured by 
special tests of reasoning,* the results, again, being cheeked by the 
teachers; and, lastly, his mental age, given directly by the present 
version of the Binet-Simon scale, unchecked and unrevised. 

“The first column of figures in Table XX. (our Table 134) 
shows the six correlations subsisting between these four measure- 
ments coupled with one another in every one of the six ways 
possible.** 

“From the six ‘total’ coefficients, taken each in turn, I have 
first of all eliminated one or other of the four factors operative. 
Prom^ the gross figures I have, by discount, found the net. The 
resulting ‘partial’ coefficients are given by the second column of 
figures in the table. A comparison of these values at once invites 
several inferences. The resemblance between the Binet-Simon 
results and the child’s school standing seems due more to the com- 
mon influence of age than to the common influence of intelligence. 
The resemblance between the Binet-Simon results and the child’s 
intellectual maturity, estimated independently, seems due more to 

* See Appendix IV., pp. 239—242. 

*With a group of nearly 300 chUdren, the probable error for correlations 
less than .12 ranges between -(-.038 and -f .039. A coefficient under .07, there- 
fore, has little or no significance; one over .11 may be received as trustworthy. 
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the common influence of school standing than to the comip^u influ- 
ence of age. The estimates for intellectual maturity owe their 
correlation with school standing — a correlation by no means I igh 
even at the outset — chiefly, but not entirely, to the common influ- 
ence of age. Wlien the influence of intelligence is excluded, there 
still remains a correspondence between age and position in school 
that is unexpectedly — indeed, I apprehend, unwarrantably — close : 
promotion goes suspiciously with seniority. The negative correla- 
tion between school standing and intelligence, obtained when dif- 
ferences in Binet age are eliminated, may seem odd; but even were 
it larger than it is, it would not be at all inexplicable.® In a group 
homogeneous in regard to mental age, children who are older 
chronologically would, in a teat measuring inborn intelligence rather 
than mere mental growth, appear duller; yet, because they are 
older, the school system elevates them to a somewhat higher class. 
Hence the paradox of a group whose mental age is uniform; the 
higher the class, the duller the child. 

TABLE 134. (Table XX of Burt). 

Observed and Partial Cokredations Between Age, Intelligence, School 
Attainments, and the Blsults of the Binet-Simon Tests. 


Partial Partial 

Factors Observed Factor GoefScient Factors Coefficient 

Correlated Coefficients Eliminated (First Eliminated (Second 

Order). Order). 


B Tests and 


Intelligence 

.78 

Intelligence 


School Work 

.91 

Ago 

.68 

and Age 

.61 

B Tests and 


School Work 

.58 

School Work 


Intelligenco 

.84 

Age 

.65 

and Age 

.56 

B Tests and Ago 

.83 

School Work 

.19 

School Work 



Intelligence 

.62 

and 






Intelligence 

.13 

School Work and 


Tests 

-.06 

Tests and 


Intelligence 

.75 

Age 

.40 

Age 

-.07 

School Work and 


Tests 

.49 

Tests and 


Age 

.87 

Intelligence 

.73 

Intelligence 

.49 

Intelligence and 


Tests 

.01 

Tests and 


Age 

.70 

School Work 

.15 

School Work 

.05 


“Let US now examine the partial coefficients of the second order, 
coefficients, that is, obtained where two factors have been cancelled 
in succession (last column of Table XX). 

“Intelligence, it may be remembered, was observed to correlate 
with the Binet tests by .84 and with school attainments by .75. 
Mediated solely by intelligence, therefore, a correlation between the 
Binet estimates and school attainments could be predicted amount- 
® Tho coefficient in question is barely twice its probable error. 
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ing at least to .75 X .84, that is, .63. The total correlation found, 
however, was as much as .91. The excess is due, in part at least, 
to the second common factor of age. But, on eliminating also the 
effect of age, there is still left a substantial surplus. With both age 
and intelligence constant, the ‘partial’ correlation between school 
attainments and Binet results remains at .61. Of all the partial 
coefficients of the second order this is the largest. There can, there- 
fore, be little doubt that with the Binet-Simon scale a child’s mental 
age is a measure not only of the amount of intelligence with which 
he is congenitally endowed, not only of the plane of intelligence at 
which in the course of life and growth he has eventually arrived; 
it is also an index, largely if not mainly, of the mass of scholastic 
information and skill which, in virtue of attendance more or less 
regular, by dint of instruction more or less effective, he has progres- 
sively accumulated in school. 

“The correlation of .49 between age and educational attainment, 
left after the elimination of ability both tested and observed, con- 
firms our previous suspicion of the undue influence of age upon 
school classification. The only other correlations surviving after 
the double elimination arc those between the Binet tests, on the one 
hand, and intelligence and ago respectively upon the other. 

“From the three final correlations thus furnished by the tests, 
and from the relevant standard deviations, can be calculated the 
several so-called ‘regressions,’ The regressions will indicate the 
relative proportions in which the three factors — age, intelligence, 
and school attainments — together determine a child’s achievements 
in the Binet-Simon tests. The complete equation is as follows: 

B == .54 S + .33 I -I- .11 A, 

where B = mental age according to the Binet-Simon scale, 

S == school attainments expressed in terms of educational 
age, 

I = intellectual development also measured in terms of 
years, and 

A = the chronological age. 

“Of the gross result, then, one-ninth is attributable to age, one- 
third to intellectual development, and over one-half to school attain- 
ment. School attainment is thus the preponderant contributor to 
the Binet-Simon tests. To school the weight assigned is nearly 
double that of intelligence alone, and distinctly more than that of 
intelligence and age combined. In determining the child’s per- 
formance in the Binet-Simon scale, intelligence can bestow but little 
more than half the share of school, and age but one-third the share 
of intelligence,” [Burt, ’21, pp. 181 to 183.] 

We are greatly indebted to Burt for this study, espe- 
cially for the application of the partial-correlation tech- 
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nique to the problem, but we should be very cautious in our 
use of the facts. Small differences in the observed correla- 
tions may make large differences in the partials. These 
small differences may be in whole or in part due to sam- 
pling error, or to differences in the amount of chance error 
in the original measures producing differences in the atten- 
uation. Suppose, for example, that Burt^s Reasoning Tests 
really measured exactly the same ability as the Binet, but 
with a greater chance error (the two reliability coefficients 
being .95 and .80) and that the true correlations for a very 
large population, each person being perfectly measured, 
were as shown in Table 135. 

If the facts were so, I and B would measure identical 
abilities and the regression equations, B = uS + bl + cA 
and I — dS -f eB 4- /A would be identical save that B and I 
would change places. 

Yet all that is required to produce Burt ^s six coefficients 
from those of Table 135 is that the self-correlation of Burt’s 
determinations of I be .15 points lower than that of his de- 
termination of B (somewhat lower it almost certainly is), 
and that, by the sampling error of his 300 cases, his corre- 
lation of B with I be .03^ too low^ his correlation of S with I 
be .09 too low, and his correlation of I with A be .06^ too 
low. There is one chance in fifty for the first of the three, 
one (diance in 5,000 for the second, and one chance in sev- 
enty for the third. The chance of all three occurring to- 
gether is very small (though by no means so small as 
1/50 X 1/5000 X 1/70, since the same sort of cases that 
would lower ra, would tend somehat to lower rsi and r^i also), 
and there is an equal chance that 30,000 cases in place of 300 
would strengthen Burt’s argument. Indeed, we agree with 
Burt that score in the Binet series is influenced by school 
training, and probably to a greater extent than score in 
Burt’s reasoning test series will be. We very much doubt, 
however, that the amount* of influence is so large in the one 
case or so small in the other as the second-order partials of 
.61 and — .07 indicate. 
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It is perhaps worth while to compute what these partials 
would be if we assume that all of Burt’s coefficients are 
valid, but assume in addition that the self-correlations of 
his four measures with a second set of independent mea- 
sures of the same qualities or abilities^ are .95 for B, .80 

TABLE 135. 

The Intbrcorrelations op One Binet Test (B), One Burt Beasoninq Test 
(I), One Measure op Schoou Work (S), and Age (A) by Certain 
Assusiftions Concerning the Intercorrelations Ip an Inpi- 
NiTE Number op Such Tests Had Been Used. 



1 

Taoco 

2 

3 

r,i by Burt 

4 

Difference (2—3) 

BS 

.96 

.91 

.91 

0 

BI 

1.00 

.87i 

.84 

.03i 

BA 

.85 

.83 

.83 

0 

SI 

.96 

.84 

.75 

.09 

SA 

.89 

.87 

.87 

0 

lA 

.85 

.76i 

.70 

.06i 


for 1, 1.00 for A, and .95 for S. They will be : 

^Ds.iA .91 and rig jj^ — .79. 

Finally, as further evidence of the need of caution in 
arguments from small differences between correlations, let 
us apply the well-known attenuation formulae to Burt’s six 
correlations, and obtain their answers to certain questions. 
The first is : What is the correlation betv/een (a) whatever 
is common to the ability measured by the Binet and the 
ability measured by the Burt Test, and (b) whatever is 
common to School work and Age? The answer is: 


V.83 X .91 X .87 X .75 


or .98 ; that is, almost everything. 

V.84X.87 

The second is: What is the correlation between (a) what- 


7 To prevent possible ambiguity, we add that we mean, not repetitions of 
the particular Binet and Burt tasks, but a set drawn at random from the same 
general collection of tasks from which the Binet Tests may be considered 
a random draft, and a set drawn from the same general collection of tasks from 
which the Burt Tests may be considered a random draft. 

31 
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ever is common to the ability measured by the Binet and 
School work and (b) whatever is common to the ability 
measured by the Burt Test and to that measured by i ,gef 
The answer is: 


V-84 X .83 X .75 X .87 
V.91 X .70 


or 1.02 ; that is, everything. 


We obtain a similar answer (1.03) to the third question 
concerning the correlation between whatever is common to 
Binet and Age and whatever is common to Burt and School 
work? 

These answers are absurd. Not the most passionate 
adherent ol one general ability as the cause of all possible 
excellences would assert that whatever is common to the 
ability to score well in Binet and Burt is also common to 
Age and School Attainment. Neither Burt nor any other 
competent psychologist would entertain the notion that 
what is common to success in the Binet and success in 
School is the same thing that is common to success in the 
Burt and Age. 

Such fantastic perfect correlations between common 
factors sometimes are due to a statistical fallacy, there 
being really nothing common in either pair of traits; but 
Dr. Burt will be too sagacious to plead this in the present 


case. 

The fact is that when a number of traits are somewhat 
nearly equally intercorrelated, as is the case with B, I, S, 
and A in Burt’s 300 pupUs, partial correlations and in- 
ferred correlations between common factors will often show 
queer, not to say absurd, results. Both procedures are of 
very great value, but they are very sensitive to the influ- 
ence of the ^‘errors” due to measuring traits in too few per- 
sons and with too few tests per person. 

The partial correlations computed by Burt for the in- 
fluence of the separate features of school work seem to us 
in better harmony with the view that B and I are two tests 
differing moderately in their susceptibility to school train- 
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ing than with his view that B (for persons alike in I) is 
very susceptible and I (for persons alike in B) not at all so. 
They are : 


TABLE 136 (XXI of Burt). 

Observed and Partiai^ OoRRELA'fiONS between the Binet-Simon Tests and 
Attainments in the Sevebaij School Subjects. 


Partial CoefScients 



Observed 

Ck>e£Scients 

(Age and Intelligence 
Eliminated) 

Composition 

.63 

.32 

Beading 

.54 

.26 

Dictation 

,52 

.21 

Arithmetic (Problems) 

.5o 

.07 

Arithmetic ( Mechanical ) 

.41 

.15 

Writing 

.21 

.01 

Drawing 

.15 

-.08 

Handwork 

.18 

-.06 


It is easy to see how Composition might be a better 
symptom than Dictation and Arithmetic of those parts of 
intellect which 13 measured but I failed to measure, but 
hard to see how training in it could improve B more than 
training in Dictation and Arithmetic did. 

Burt’s study, then, though it is by one of the most ex- 
perienced and able workers in this field and is the most ex- 
tensive and the most searching study of the problem that 
we have found, does not convince us either that his Rea- 
soning Tests, when given to a group identical in age and in 
ability measured by the Binet, will measure original capac- 
ity uninfluenced by school advantages, or that the Binet, 
when given to a group identical in age and in the abiUty 
measured by his Reasoning Test score, will measure a com- 
posite made up, half or more, of school advantages. If we 
knew of any test that did the former, we should have re- 
peated his experiments testing out both the Binet and the 
Burt with the aid of the test of sheer original capacity 
instead of making this long counter-explanation of his 
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results. But our general study of the problem has not re- 
vealed any such test. To this general study we may now 
return. 

THE USE OF ALTITUDE AND WIDTH OF INTELLECT 

According to the orthodox views of what original nature 
is likely to contribute and what the environment is likely to 
contribute, it would be reasonable to choose the altitude of 
intellect and the width W(1C or lA or IV or ID, etc.) 
as the two extremes, the area A(1C or lA or IV or ID, etc.) 
being intermediate in its causation. It seems, at least, much 
easier for a good home or school to increase the number of 
easy things which a child can do than to enable him to do 
harder things than he has ever done. Dull men who could 
never learn to use indirect discourse correctly in Latin can 
learn the easy features of a score of languages. A favor- 
able oxjportunity and assiduity seem to be all that are 
needed to teach anybody twice as many thousand easy ac- 
complishments as he has acquired with meagre opjjortunity 
and less study. 

This seems almost axiomatic. We were almost con- 
vinced of it until we investigated the actual relations be- 
tween altitude, width, and area of Intellect CAVD and be- 
tween the liigher selective and organizing abilities and 
the lower or associative. The correlations are such as to 
cast doubt upon the doctrine that the number of easy in- 
tellectual accomplishments which a person learns depends 
chiefly, or even largely, on the stimulus of the environment. 
On the contrary, the number which a person can learn 
seems to be limited by his nature almost as much as is the 
degree of difficulty which he can master. If, by a miracle, 
intellectual accomplishments were all of exactly the same 
difficulty, we have reason to believe that the number which 
a person had learned at a given age would show the same 
hereditary relations as are shown now by the altitude 
which he reaches. As things are, the competent intellects 
learn approximately all the easy things which the incom- 
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petent intellects learn, plus a large balance of harder things. 
The imbecile probably could not learn twice as many things 
as he now does, no matter what opportunities were pro- 
vided for him. 

The explanation of tlje difference between the expecta- 
tion from general psychology and the results shown in our 
correlations is to be found in the fact that there are rela- 
tively so few easy intellectual accomplishments. In order 
to increase the number of important things an intellect can 
do, you soon have to put it to doing harder things, because 
the easy ones are so soon used up. If there were ten mil- 
lion different single tasks at each level from 25 to 45, it 
would perhaps be possible to take two intellects of equal 
original capacity, and by certain deprivations hold one 
down to three million at level 25, and by certain advantages 
stimulate the other to reach four or five or six million at 
that level. And doubtless any wise psychologist would con- 
sider this attempt under these conditions much more prom- 
ising than the attempt by an equal difference in depriva- 
tions and advantages, to hold one down to inability to do 
anything harder than 25, while the other was made able to 
do tasks at 35 or 40. But the conditions are unreal. To 
push the second intellect's score up by twm or three million 
would, as things are, mean to have him master tasks above 
level 25. 

Another part of the explanation is found in the fact that 
altitude as it is used in our correlations is not the same as 
the altitude meant in the statement that it is much easier 
for environment to increase the number of easy things an 
intellect can do than to enable it to do harder things than 
it has ever done. To do a harder thing means for us to do 
a certain percent of the tasks at a higher altitude, one in 
forty, or two in forty. Now, if there are twice as many 
tasks at level 31 as at level 30, the intellect which is stimu- 
lated to do 1 in 40 at level 31 has to learn twice as many 
tasks as the intellect "which is stimulated to do 16 instead of 
15 out of 40 at level 30. This is naturally harder for the 
environment or for any other force. 
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A third part of the explanation is to be found in the fact 
that our correlations represent the status as it is created by 
such differences in environment as do now act, not the 
status as it might conceivably be. If the action of the en- 
vironment is positively and closely correlated with that of 
original nature, so that the best born are the best bred, and 
the worst born the worst bred, approximately, the high cor- 
relations between altitude and A\^(1C or lA or IV, etc.) 
which we find, may not be necessary as results of the nature 
of intellect, and may be alterable, as by giving equally ad- 
vantageous training to all. Or, if the effect of difference in 
environment is now very small in comparison to the effect 
of original nature, so that the high correlations found are 
chiefly due to original nature, these may be alterable by in- 
creasing the differences in environment. 

We may then retain in a somewhat tempered form the 
expectation that environment can extend the area of intel- 
lect by adding width somewhat more easily than by adding 
altitude. 

So far as concerns the special question of a differential 
diagnosis between nature and nurture in the world as it 
now is, however, we shall receive almost no assistance by 
using altitude for the symptom of the former, width W(1C 
or lA or IV or ID, etc.) for the symptom of the latter. 
Where the wddth is not 100% or 0%, its correlation with 
altitude is too near perfection. In the world as it now is, 
they are due to almost the same causes. 

OTHER METHODS OF SEPARATING ORIGINAL. CAPACITY FROM 

ACQUIRED ABILITY 

There are two other ways of approaching the problem, 
besides that of differential tests. The first is to approxi- 
mate original nature by measurements early in life before 
environmental forces have had much opportunity to act on 
intellect. The second is to measure a man^s intellect as we 
find it and to make the best allowance we can for the favor- 
able or unfavorable action which the environment has had. 
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The former is not so fantastic as it would have been a 
few years ago. It would then have been even more absurd 
to claim to measure original capacity for intellect at three 
or four or five than to claim to measure at that age the orig- 
inal tendency to adult stature. We now have far better 
tests of intellect at low levels and in early stages ; and the 
correlations between intellect at three and intellect at 
thirty, environment being equalized, may be closer than we 
think, much closer for example than the correlations be- 
tween stature at three and stature at thirty with equalized 
environment. The contribution of original nature is all 
there in the individual at three years, or at three days. 
How much of it is revealed in external behavior, and how 
much is hidden in the constitution of the neurones, is a 
question for investigation. 

On the other hand, we have to reckon with the evidence 
and arguments now being brought forward to prove that 
the environment of the first three years is very potent. 
Freud and others contend that the trends of character are 
much influenced by the environmental forces acting in these 
years; and we may expect some of them soon to make the 
same contention with regard to intellect. 

The method of measuring original capacity by measur- 
ing attainment and making an allowance for the benefits 
and handicaps of environment, is what has been and is used 
in many scientific investigations and in practice by wise 
educators and advisers of youth. For example, of two boys 
making equal scores in Army Alpha, one from an English- 
speaking family with four years in an excellent city school, 
the other from an immigrant family and an inferior school, 
the second will be rated as the better in natural intellect 
and future intellectual promise. Its success depends, of 
course, on the adequacy of one^s knowledge of what the 
environment has been in each individual case, and on the 
wisdom of one’s theories concerning the action of environ- 
mental forces on intellect. 
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In SO far as our work improves the measurement of in- 
tellect, it will improve this sort of measurement of, or in- 
ference about, original capacity for intellect. The balance 
of the program is to improve measurements of the environ- 
mental forces which help or hinder the attainment of in- 
tellect. 


SUMMARY 

On the whole, the problem of analyzing a person’s in- 
tellectual ability into an amount due to nature and an 
amount due to nurture, is unsolved. No task or test has 
been proved to be a measure of the former alone. The wis- 
est procedure at present is to equalize environmental forces 
by using a wide variety of data with which all individuals 
have had adequate experience, and to make as correct al- 
lowances as we can for w^hat we cannot equalize. With the 
progress of science, we may hope for accurate measure- 
ments of intellect at earlier and earlier ages, and for truer 
rules for making allowances for environmental differences. 



CHAPTER XVII 


tHANGBS IN THE ALTITUDE AND ArEA OF INTELLECT WITH AGE 

It is not at present possible to distingnish at all accu- 
rately within the general gain in intellect with age from 0 to 
p years or later, the share of mere maturity, mere inner 
towth, from the share of the experience and training which 
fe implies. The reasons for this failure are much the 
same as for the failure to distinguish the shares of original 
capacity and environmental circumstances in the determi- 
nation of intellect. We lack tests which measure maturity 
by itself, and tests "which measure training by itself. We 
also lack extensive investigations using the partial-correla- 
tion technique with such imperfect symptoms of maturity 
and training as are available. In this chapter, therefore, 
age means chronological age, and whatever it involves 
under the conditions of present-day life in America. 

ALTITUDE 

The curve of altitude of Intellect CAVD with age is of 
the general parabolic form shown in Fig. 64. There is a 
rise from 0 to about 30 at 64, to about 34^ at 10^, and to 
about 36 J for adults 21 years old.^ 

1 These estimates are subject to a thorough going investigation which is 
being made by Miss Kowell, using our data and additional data obtained by 
her. They are made from the following facts: The average altitude CAVD 
of imbeciles of mental age 6—0 to 6—11 is 30.1. Taylor ['23] has shown that 
ordinary children of chronological age 6—0 to 6-11 do not differ much in such 
tasks as these, from imbeciles of mental age C.O to 7.0. The average altitude 
CAVD of pupils 10 yr. 0 mo, to 10 yr. 11 mo. in grade 4B of School X, is 
33.85. The average altitude CAVD of pupils of the same ago in grade 5A 
of School X is 33.9. The average for pupils of the same age in 5B of School 
X, is 35.15. (A is the lower half ; B is the upper half.) In School X the 
ordinary pupil 10.0 to 11.0 in age is in grade 5A or 5B. The numbers of 
ages 9.0 to 10.0 grades 4B, 5A and 5B are S6, 24 and 27 ; those of age 10.0 
to 11.0 are 39, 47 and 88; and those of age 11.0 to 12.0 are 18, 29 and 48. 
The ordinary pupil of age lOi may then be expected to score somewhat above 
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It has been shown that the score in stock intelligence ex- 
aminations such as the National, Otis, and Haggerty, is 
substantially a measure of altitude. Consequently, we may 
take the curves of these scores in relation to age as approxi- 
mate curves for the altitude of intellect in relation to age, 
first transposing the scores into terms of equal units. 

This has been done for the National A, Otis Advanced, 
and Haggerty Delta 2, with the results shown in Table 137. 
The three examinations do not agree at all closely, the Hag- 
gerty official norms being especially divergent. Scores in 
equal units are available for all for the interval from age 
10 to age 15. The gain from age 10 to age 11 is 28 percent 
of the gain from 10 to 15 in the case of the National; 25 i>er- 
cent of it in the Otis, and 20 percent of it in U(l), and 33 
percent in H(II) . The gain from 11 to 12 is 27, 22, 17^, and 
31 percent of the gain from 10 to 15 in the National, Otis, 
H(I), and H(ll), respectively. Corresponding percents 
for the gain from 12 to 13 are 25, 18, 15, and 35; for the 
gain from 13 to 14 they are 15, 18, 21, and 6 ; for the gain 
from 14 to 15 they are 5, 17, 261, and — 5. 


The discreijancies are about as great if the original 
scales are used, the percents of the gain from 10 to 15 then 
being : 


Age 

National A 

Otis 

Advanced 

Haggerty 
Delta 2 
OfScial 

Haggerty 
Delta 2 
Madsen 

10-11 

28.3 

23.6 

18.3 

33.4 

11-12 

26.7 

21.8 

18.3 

30.9 

12-13 

25.0 

18.2 

16.7 

34.3 

13-14 

15.0 

18.2 

21.7 

5.6 

14-15 

5.0 

18.2 

25.0 

-4.2 


the average of 33.9 and 3J.159 which is 34.53. As a check on the estimate of 
34i or more for age 10, we have the fact that the 86 nine year olds in grade 
4B have a median score of 34.0. In this school the ordinary 9 year old has 
reached grade 4B or, less often, 4A. The average altitude GAVB of the 44 
adult recruits in the United States Army is 36.5. These are enlisted men 
chosen for training in the Signal Corps, all but one between 18 and 25 years 
old, whose median amount of schooling is grade 8, and who may therefore be 
taken to represent the median of the white population, 18 to 24 years old, or 
a point a little above it in intellect CAVD. 
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There is evidently need for a careful critical study of 
these facts. If the three examinations measure the same 
thing, and if the averages or norms for the ages 10 to 15 
^re correctly determined, the curves in relation to age 
should be the same. 

TABLE 137. 

THB BZI.ATION OF SCOBES IN NaOTONAL A, OtIS ADVANCED, AND HAOOEBXT 

Delta 2, to Aob. H (I) Befebs to the Official Aob-nobms; 

H(II) Refers to the Madsen Results. 


Age 

Scores by the original scale 
Nat. Otis H(I) H (II) 

Scores by a scale of equal units 
Nat. Otis H(I) H(II) 

S 

46 

25 

25 

38.8 

51.2 




9 

Cl 

40 

43 

56.6 

64.5 

38.6 


57.8 

10 

76 

55 

55 

66.8 

78.0 

54,9 

56,1 

68,8 

11 

93 

68 

66 

78.7 

94.0 

68.6 

67.9 

79.6 

12 

109 

80 

77 

89.7 

109.5 

80.6 

78.1 

89.7 

13 

124 

90 

87 

101,9 

124,0 

90.3 

87.1 

101.3 

14 

133 

100 

100 

103.9 

132.4 

100.0 

99.3 

103.3 

15 

136 

110 

115 

101.8 

135.4 

109.5 

114.8 

101.8 

16 


120 


106.7 


119.0 


106.9 

17 


127 




125.7 



18 


130 




128.6 



19 


130 




128.6 



Gains 

By tho original scale 

By the scale with equal units 


Nat. 

Otis 

H(I) 

H(I1) 

Nat- 

Otis 

H (I) 

H(II) 

8-9 

15 

15 

18 

17.8 

13.3 




9-10 

15 

15 

12 

10.2 

13.5 

16.3 


11.0 

10-11 

17 

13 

11 

11.9 

IG.O 

13.7 

11.8 

10.8 

11-12 

16 

12 

11 

11.0 

15.5 

12,0 

10.2 

10,1 

12-13 

15 

10 

10 

12.2 

14.5 

9.7 

9.0 

11.6 

13-14 

9 

10 

13 

2.0 

8.4 

9.7 

12.2 

2.0 

14-15 

3 

10 

15 

- 2.1 

3.0 

9.5 

15,5 

- 1.5 

15-16 


10 


- 4.9 


9.5 


- 4.9 

16-17 


7 




6.7 



17-18 


3 




2.9 



18-19 


0 




0.0 




The disagreement between the Haggerty official age- 
norms and Madsen’s results is especially noteworthy be- 
cause Madsen’s selection of 14-year-olds and 15-year-olds is 
presumably of the superior, the duller ones being more 
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likely to leave school. The average gains from 13 to 14 and 
from 14 to 15 for all children would consequently tend to 
be even less than those for his groups. 

The general drift of these determinations is toward a 
parabolic curve of the form of Fig. 64. If the two sets of 
determinations (by equal-unit scores) of the Haggerty 
Delta 2 are combined with equal weight, and if the three 
curves for the three examinations are then combined with 
equal weight, the rise in the ordinates from 10 to 11, 11 to 
12, 12 to 13, 13 to 14, and 14 to 15 are in the proportions: 
26.5, 24.4, 22.7, 15.5, and 10.75. 



Fio. 64. The g'eiioriil nature of the relation of altitude of intellect to age 

in years, 0 to 20. 


These results for altitude of intellect in relation to age 
may be compared with those attained by Brooks [’21], in 
his careful and extensive experiments. In general our work 
corroborates his. The gains in what he terms the ** higher” 
functions from 10 to 15 by years according to his final com- 
bined table [’21, p. 68], allowing equal weight to the boys 
and to the girls are in the proportion 24.8, 19.7, 19.4, 18.8, 
and 17.4. These gains may all be unduly large and the later 
gains at later ages unduly large in comparison with the 
gains at earlier ages, because of the practice effect which 
retesting involves. Data are not available to correct for it. 
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Probably an allowance of one-twelfth of the gains from age 
10 to age 15 would be considered enough by all students of 
the matter, and too much by a majority of them. If this 
veiy large allowance is made, the gains are in the propor- 
tions : 28.2, 19.5, 18.9, 17.9, and 15.5. Neither in our CAVD 
results, nor in the National-Otis-Haggerty estimate, nor in 
Brooks’ results is there any justification for the doctrine 
that the gain in altitude of intellect of the sort measured 
by existing intelligence tests is zero after 14, or after 15, or 
even after 16. It decreases, but it should not become inap- 



Fig. 65. The relation of area of intellect to age in years 0 to 20, assuming 
Fig. 64 as correct, with a slight increase In the number of tasks with 

increase in difficulty. 

preciable until 18 or later. According to our results the de- 
crease from 14 to 18 is not an abrupt slowing up of a gain 
that has been steady hitherto, but is part of a general nega- 
tive acceleration which began long before the age of 6J. 

ABEA 

The form of the curve for area of intellect in relation to 
age is not known even approximately for CAVD or any 
other specified intellect, since the number of tasks at each 
altitude is not known. The arguments presented in Chap- 
ter XII make it highly improbable that the curve for area 
is like Fig. 64 with a rapid rise at the lowest ages and 
decreasing annual increments thereafter. A very moderate 
increase of the number of tasks with increasing difficulty is 
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sufficient to make the area curve from 0 to 16, one with 
creasing annual increments as in Fig. 65. 

GENERAL OONSIDEBATIONS 

The limitations of the special tasks used in CAVD, Na- 
tional, Otis, and the like, should be kept in mind in all 
thought concerning the relation of either altitude or area 
of intellect to age. The verbal and mathematical tasks 
which built so largely in these examinations may be more 
like those which occupy the intellects of children from five 
to fifteen than those which occupy the intellects of young 
people from fifteen to twenty-five, or those which occupy 
the intellects of men and women from twenty-five to thirty- 
five. It is conceivable and probable that the person who 
ceases to improve in altitude CAVD may continue to im- 
prove ill altitude Bu, Ch, Ho, So. (Business, Child [Manage- 
ment, Houst'hold [Management, and Social Arrangement). 

It is also the case that after a person acquires a certain 
amount of general linguistic and mathematical ability, and 
of general information about the sort of things which every- 
body is ashamed not to know, he usually devotes his mental 
abilities to the specialized abilities useful in his trade, busi- 
ness, or profession, hobby, and social circle. The correla- 
tion between CAVD (or any similar ability) and such spe- 
cialized abilities is doubtless high, but it is probably not 
perfect; and these specialized abilities may begin their 
rapid rise in altitude at an age when CAVD altitude has 
almost ceased to gain. 
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SuMMABY or Results and Applications to the Measube- 
MENT OF Human Abilities in Genebal 

SUMMABY OF BESULTS 

All the measurements of intellect which have been made 
hitherto and which psychologists may expect to make in 
the future, unless means are found of defining and counting 
units of connection in the neurones, are inventories. They 
are records of the degree of success in accomplishing intel- 
lectual tasks. If all intellectual tasks are listed and ar- 
ranged in levels of difficulty, the inventory may be syste- 
matized into a record of how many the intellect in question 
can do at each level and how quickly he can do them. From 
the record of how many it can do at each level, three useful 
measures may be abstracted. One is altitude, that is, the 
degree of difficulty at which a given percentage of success 
is attained. The second is width, that is, the percent of suc- 
cesses at any given altitude or the average percent of suc- 
cesses at any given series of altitudes. The third, which 
may be called area, is the total number of tasks done cor- 
rectly, or the percentage which this total is of the number 
of tasks in the entire list. 

An intellectual task is one, success in which depends 
upon all of intellect and nothing but intellect. Intellect is 
definable by a series of tasks, and we have so defined one 
variety of it. Intellect CAVD, and could so define any other 
variety of it. A CAVD intellectual task is, then, one suc- 
cess at which depends upon all of Intellect CAVD and 
nothing but Intellect CAVD. Tasks can be devised which 
do substantially meet this requirement, success at one of 
them correlating perfectly (or as closely as its own self- 
correlation permits) with success in the entire series. 
CAVD intellect is nearly or quite homogeneous in the sense 
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that the ability which determines success at any one, level 
of difficulty is, to a close approximation, simply a larger or 
smaller amount of the same ability that determines success 
at any other level of difficulty. The evidence for this is the 
very small and uncertain reduction of correlation as more 
and more remote levels are taken, and the substantially per- 
fect correlation between score at any one level and the sum 
of the scores at all levels, or any other score representing 
in any reasonable way ability at the entire series. 

No short single task, however, can be measured in re- 
spect of its intellectual difficulty, for no short single task 
can be devised which depends for its success upon all of in- 
tellect and nothing but intellect. Even composite tasks 
made up of forty or more single tasks well selected to repre- 
sent Intellect CAVD (or any other specified sort of intel- 
lect) are not perfectly intellectual and must be treated as 
measuring intellect plus an error for which allowance must 
be made in all inferences from measures of their difficulty 
to measures of their ini ellectnal difficulty. With short 
single tasks, this error becomes so large and so variable 
amongst different tasks that no trustworthy allowance can 
be made for it. 

The difficulty of a task and the difference in difficulty 
between one task and another may be measured by the per- 
centages of certain grouijs which succeed with it. For it has 
been demonstrated that the form of distribution of a school 
grade population from G to 13 in respect of altitude of in- 
tellect is to a very close apiDroximation that of the normal 

1 - 

probabilitj’^ surface, defined by y= =e The diffi- 

culty of an intellectual task may also be measured by the 
percentage of successes among the various trials of the 
same individual, or by the average of such percentages 
from any given number of individuals of the same general 
degree of intellect, since it has been demonstrated that the 
varying conditions of an individual from time to time 
(omitting such extreme conditions of sleep, illness, intoxi- 
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23 

261 

281 

291 
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311 

32 
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341 

361 

371 

381 

391 

40 

411 

41 } 

43 


ed, to Q, at which only 23 percent of college graduates 
eceed. The differences in difficulty are determined ap- 
oximately. They can be determined to any given degree 
precision by the methods outlined. 

The distance from the difficulty of Task A to the diffi- 
ty of Tasks 36 and 37,^ which are at or near the absolute 
ro of intellect, has been determined by a consensus of ex- 
rts as about 4.35 times the difference between Task A and 
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of intellectual difficulty from an absolute zero in equal units 
as shown below. This scale is at all points more accurate 
than the best scales previously available ; and 
is accurate enough for many scientiAc an^ 
practical uses from I to Q, covering the inteij*- 
val from the upper extreme of the feeble- 
minded to the 98 or 99 percentile adult intel- 
lect. It should, however, be improved bjy 
more extensive experimentation. 

If alternative CAVD composites are con- 
structed to be like A, B, C, etc,, respectiveliy, 
in difficulty, but different from them in con- 
tent, until all possible CAVD tasks are used, 
it will be found that the number is not the 
same at each level. In general, the easier the 
level, the smaller the number of CAVD tasks 
that can be made without using the same 
single task twice. 

If the number of tasks possible at le’ el 1 
to 5 is a, the number possible at levels 6 to 10 
is a -f- b, tlie number possible at levels 11 to 
15 is a + b -f- e, and so on. If the tasks of our 
series are represented by the column of Pig. 
60, the total number which could be con- 
structed and which are exemplified or sam- 
I)led by A, B, C, etc., would be represented by 
a figure vrith a top very much wider than its 
bottom, as in Pig. 67, or Pig. 68, or Pig. 69. 
When the exact shape of this surface of fre- 
quency of intellectual tasks acording to diffi- 
culty is determined, the width and area of in- 
tellect can be measured. The measurement 
of an intellect should then be a measurement 
of its success or failure with each of a series 
of composite tasks each of which depends for 
success substantially on all of intellect and nothing but in- 
tellect. If the number of these tasks in the scale used to test 


Fia. 66, The 
pattern of area 
of intellect in a 
sampling of N 
tasks, at each 
level of diffi- 
culty. 
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the intellect in question is proportional at each level to the 
number of such tasks that the world offers, the width of the 
intellect in question at each level and its area are given in 
the record. If the number of tasks in the scale is not pro- 
portional to the total existing number in this way, the num- 
bers of the record at each level must be multiplied by suit- 
able factors to obtain widths and area. In either case the 
total record for a person whence altitude, widths and area 



Fia. 67. 

are derived may be with high probability inferred from 
the scores in three 40-coniposite tasks where the percent of 
single tasks right is near fifty. 

Such a measurement is fundamentally right, and im- 
proved varieties of it are all that can be exiiocted on the 
level of external behavior ^vith voice, paper, and pencil, and 
the like. The physiological facts in the neurones which pro- 
duce and, in the deeper sense, are intellect, are not known. 
When they become known, they may or may not be amen- 
able to observation and measurement. 

As a result of the high correlations found between mere 
associations or connections and the so-called ‘‘higher’* 
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Fi<j. 69. 

Figs. 67, 68 and 69. Samples of probable relations of number of ta^ to 

level of difficulty. 



ST7MMABY AND APPLICATIONS 


475 


processes of abstraction, generalization, organization and 
control, inference and reasoning, we have advanced the 
hypothesis that the original basis for altitude, width, and 
area of intellect is the mere number of possible connections 
in the neurones whos'e connections correspond to having 
and using ideas. If this hypothesis is verified, it may be 
possible sometime to discover means of counting the num- 
ber of possible connections in distinction from the number 
which actually function, and so of distinguishing original 
intellectual capacity from acquired intellectual ability. So 
far as tasks for external behavior are concerned, there are 
none which measure original capacity for intellect uninflu- 
enced by training. One can only measure intellect, and then 
make such allowances for advantageous and for disadvan- 
tageous training as are shown to be reasonable. 

Attaining any specified score in a stock intelligence ex- 
amination such as the Stanford Binet or Army Alpha or 
National may best be considered as succeeding with a cer- 
tain more or less intellectual task. That is, to attain 190 in 
Army Alpha under standard conditions is to succeed with a 
certain task; to attain 185 is to succeed with a certain 
easier task, and so on. With this interpretation of the 
scores, a scale in equal units can be worked out for any 
such test over most of its range by suitable experimenta- 
tion ; and this was done for many of these examinations. 

Using equal-unit scales, the form of distribution of in- 
tellect in a group of the same age was determined for ages 
10, 11, 12, 13, and 14, and by inference for younger ages. 
The results of the determinations for adults were incon- 
clusive, those from Army Alpha and those from Army Ex- 
amination a being very different. 

The change in altitude of intellect with age is obviously 
characterized by negative acceleration, the curve being 
roughly as shown in Fig. 64. The change in area of intel- 
lect, however, will show a very different course. Even with 
an extremely conservative estimate of the increase in the 
number of intellectual tasks as difficulty is increased, the 



476 


THE MEASUREMENT OP INTELUEGENGE 


additions to area of Intellect CAVD are larger from 4 to 8 
than from 0 to 4, and larger from 8 to 12 than from 4 to 8, 
in the case of children with ordinary school facilities. 

APPLICATIONS TO THE MEASUREMENT OF HUMAN ABILITIES IN 

GENERAL 

The principles and techniques which we have developed 
for the measurement of intellect are, with minor modifica- 
tions, suited to the measurement of a very large proportion 
of human abilities. 

Consider the following list, which is representative of 
traits which either have been measured in the last twenty 
years or have been suggested as traits which, for theoreti- 
cal or practical reasons, it would be desirable to measure, 
and includes sensory, sensori-motor, academic, moral, 
social, and economic abilities: 

Hearing. 

Discrimination of pitch. 

General motor skill. 

Ability in assembling mechanisms. 

Ability in drawing. 

Ability in the written use of the vernacular. 

Ability in spelling. 

Honesty about money. 

Honesty about cheating. 

Popularity. 

Ability in carpentry. 

Ability in salesmanship. 

A consideration of each case will show that with occa- 
sional exceptions the following principles and techniques 
are applicable: 

1. What is measured is a product produced, a task 
achieved. This may seem somewhat far-fetched in the case 
of hearing, honesty, and popularity. In hearing, the pri- 
mary product produced is within the nervous system, being 
evidenced by the person’s awareness of sound, but from 
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the point of view of measurement this is evidenced by the 
correctness of his ‘*Yes*^ and products as the trials 

are made with the sound and with no sound. In honesty, 
the products produced include acts of 'not doing as well as 
acts of doing certain things. In popularity the primary 
products are in the nervous systems of other people, but 
these are measured by smiles, votes, loans, companionship, 
and favors of various sorts. 

2. The measurement of any of the products produced 
involves valuation. In the case of hearing, discrimination 
of pitch, and honesty about money and about cheating, the 
process of valuation is included in the definition of the abil- 
ity. For hearing, to hear is obviously better than not to 
hear. For discrimination of pitch, knowing which of two 
tones is higher is obviously better than not knowing. For 
honesty about money, not stealing is obviously better than 
stealing. The question of valuation does exist, but it has 
been settled by the statement of the kind of product to be 
produced. 

3. In measuring intellect, ive favored the arrangement 
of tasks so that the score could he success or failure, though 
we carefully loft room for a scale of credits for various de- 
grees of goodness’^ in the accomplishment of an intellec- 
tual task. In the cases of drawing, written composition, 
ability in carpentry, and to a less extent in some of the 
other abilities, the arrangement of many tasks each for a 
two-compartment score may not be so effective as the ar- 
rangement of fewer tasks each for a score graduated in 
perhaps fifty or more compartments. How to make full 
use of such graduated scores and still obtain intelligible 
measures of difficulty, range and speed then becomes a 
problem. We may best defer our answer until we have 
considered the next two principles. 

4. The measurement of the ability is in essence an in- 
ventory. We can satisfactorily define the ability only by a 
list of the products which it produces — the tasks wMch it 
achieves. We measure it only by measuring a sample 
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which represents this total series. Even the case of hear- 
ing is a case of an inventory, though a relatively simple one. 
A person’s hearing is not equally good at all pitches, and 
there may be other complexities. Discrimination of pitch 
may vary with the intensity and timbre of the tones. 

5. The tasks in such an inventory {a) vary in difficulty'^ 
and (h) may he usefully considered as varying in difficulty 
for hearing (or for discrimination of pitch, or for whatever 
the ability is) as a totality, or for such parts of the ability 
as the task requires, with more or less intermixture of diffi- 
culty due to other factors than hearing (or whatever the 
ability is). That is, we may usefully entertain the abstract 
conceptions of difficulty for all of hearing ability and noth- 
ing but hearing ability, difficulty for all of ability to dis- 
criminate pitches and nothing but that ability, difficulty for 
all of motor skill and nothing but motor skill, etc. 

This should be unquestioned for hearing, discrimina- 
tion, motor skill, assembling, and spelling. But in the case 
of the other traits in our list, it is not so clear that the tasks 
form a graded series in difficulty, and it is much less clear 
that it will be useful to apply the conception of difficulty 
for all of a certain ability and nothing but it, and of alti- 
tude as a feature of the ability which is measured by the 
degree of its kind of difficulty at which it can succeed. 

For example, we do not often think of increases in abil- 
ity in English composition as the achievement of harder 
and harder tasks, but rather as the production of better 
and better products. This procedure can, however, be put 
into conformity with the general plan which we have 
adopted for intellect. All that is needed is to define a 
certain degree of difficulty as the difficulty of producing a 
product of a certain excellence. Thus when a person 
writes a composition on “Pishing” which is scored as 64 
on the HUlegas scale, we may regard him as having at- 
tempted the following tasks with the stated results. 

^DilBculty being defined for some specified group by the percentage of 
failures at the task in question* 
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To write a composition on “Fishing” of quality 
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This is a part of the answer to the problem raised in 
connection with the third principle, providing a method of 
utilizing graded-credit scores and still retaining the ad- 
vantages of the “success-failure” scoring. It could have 
been utilized in our completion tasks, scored as 3, 2, 1, or 0, 
by calling a 3 a success at one level and a 2 a success at a 
lower level. In using it we should bear in mind that 
graded-credit scores for one task usually have a consider- 
able subjective element^ and may need some special pro- 
visions to eliminate the errors thereof. We should also 
bear in mind that nothing is added to the reliability or 
weight of a determination by re-stating it as a series of 
successes and failures. To replace “scored 64 by a com- 
position,” by “scored 40S, 50S, 60S, 70F, 80F, 90F by a 
composition,” or to replace the latter by “scored 30S, 35S, 
40S, 50S, 55S, 60S, 65F, 70F, 75F, 80F, 8r>F, 90F by a com- 
position,” gives no added reliability or weight, so long as 
it is the same composition scored by the same persons. 

We do not have to abandon measurement by the qual- 
ity of the product measured by a gradation of credits, if 
we institute measurement by degree of difficulty mastered. 
The latter may be added without displacing the former. 

In the case of drawing, the latter is already in good use. 
We not only rate products on scales of general and special 

3 This is so unless they are made up by putting together credit points for 
specified objective features in the product. In that case they are better treated 
as summations of S’s in separate tasks, to wit, the pToduction of those objec- 
tive features. 
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merit, but also set series of tasks graded in diificulty. In 
the case of honesty about money and about cheating, the 
concept of difficulty is applicable. It is more difficult to 
be honest in paying a railroad than to be honest in paying 
a newsboy. Many more men will cheat the government 
than will cheat their partners. In the case of popularity, 
the concept is applicable. Indeed, when we say that John 
is much more popular than James, we usually mean pre- 
cisely a combination of “altitude” and “width,” that, for 
example, John is more liked than James by the people who 
like both,* and that John is liked by more people than 
James is at any degree of liking that we may choose to 
take. In such a case the degrees of difficulty are furnished 
by the differences in persons and likings, it being “harder” 
to win much liking than little from the same person, and 
to win the liking of the least friendly people than that of 
the most friendly. Carpentry and salesmanship are much 
like drawing and the written use of the vernacular in re- 
spect of the applicability of the concept of difficulty. We 
can measure variations in the ability either by a graded 
scale of credits for the quality of the product, or by^a series 
of tasks graded in difficulty; and the second method can 
be operated with no loss to the first. 

Whether in any case it is desirable to operate measure- 
ment in terms of the point on a scale consisting of tasks 
graded for difficulty, where 50% S (or some other assigned 
percentage of successes) is reached, depends in large mea- 
sure upon the usefulness of the attempt to abstract out 
difficulty for, say, ability in carpentry in its entirety and 
untainted by anything other than ability in carpentry, and 
measure in terms of it. 

In the past, we have not tried to distinguish at all pre- 
cisely the ability in carpentry from other abilities. If a 
thousand boys made chairs, we have used the quality of 
the chairs which they made as a measure of their several 
abilities in carpentry, despite the fact that John may have 

^ Or that certain people like John better than any people like James. 
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succeeded chiefly by general carefulness, and James may 
have failed by inability to read the instructions, and Tom 
may have won his standing largely by the artistic excel- 
lence of some carving and painting which adorned his 
chair, and a dozen other§ may have won their standings, 
each by a different compound of abilities. In the case of 
intellect, it is well worth while to seek rigorous measures 
of intellectual difficulty, because intellect is so important 
an ability and because altitude of intellect turns out to be 
a fairly unified, coherent variable properly represented by 
cardinal numbers. But will this be so with the altitude of 
ability in carpentry? How many thousands of abilities 
are we to assign specific difficulties and altitudes to? A 
girl makes a dress, and we easily thereby have some mea- 
surement of her ability to make that sort of a dress. How 
far shall we go in our use of this task of making a dress as 
a measure of difficulty for ability in sewing, difficulty for 
ability in design, difficulty for motor skill, difficulty for 
executive ability; difficulty for honesty in not obtaining 
illegitimate help, and so on and on? 

The general answer is that it is desirable to settle on 
the most fundamental and mutually exclusive abilities and 
learn to measure them first ; that many of the thousands of 
abilities with which the sciences and arts of man deal are 
doubtless compounds or derivatives of more elemental 
abilities; that most of them are far from being mutually 
exclusive; that new professions and trades and topics for 
study add to the number of these abilities without any alter- 
ations in the fundamentals of human nature ; and that con- 
sequently science may often refuse to measure the altitude 
of, say, ability in carpentry in its entirety and uninfluenced 
by anything save ability in carpentry. Ability in car- 
pentry may be left to a looser definition and cruder mea- 
surement. If, however, science does undertake to make a 
true measure of ability in carpentry, it is possible to apply 
the concept of specific difficulty for ability in carpentry. 
For the most satisfactory measurement it is necessary to 
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do SO. The same will hold of other mental abilities, with 
few exceptions. 

6. There are usually more ta^ks than one at each level 
of difficulty f so that the range or width of the ability at any 
level may he measured by the ^percentage correct at that 
level, and, if desired, a measure of surface may be made by 
summing the widths at all levels. 

This is in a sense a corollary of principle 4. It needs 
no comment. 

7. Measurements of speed should be of the speed of 
successful performances. Nothing useful concerning an 
ability is measured by the time required to fail at a task.^ 

8. Width being measured by number and speed by time, 
the technical problem with any of these abilities is to pro- 
vide a scale for altitude, that is, for its specific difficulty. 
Knowledge of the differences in diflficnlty (for the ability 
in question) of a scries of tasks, and of their differences 
from an absolute zero of that difficulty, are the two desid- 
erata in such a scale. 

9. Knowledge of the differences in difficulty (for the 
ability in question) may he had if ( a) the form of distribu- 
tion of the varying conditions of the ability in an individual 
is known; or if (b) the form of distribution of the varying 
abilities of the individuals in a group is known. These 

9 The relative amounts of importance of altitude^ width, and speed in the 
measurement of an ability vary. Consider, for example, drawing, spelling, and 
typewriting, asking the three questions, ^‘How difficult a taskf many 
tasks at easy levels? and ''How quickly? In drawing, altitude is of 
prime importance, since if a person can do the hard tasks, that is, can draw 
well, he can usually do, or very quickly learn to do, nearly all of the easy 
tasks, that is, draw identifiable cows, chickens, houses, trees, and the like. In 
spelling, width is of especial importance, since the ability to spell very difficult 
words is less valuable for the world's welfare or as a symptom of excellence, 
whereas failure with an easy word is an annoyance to many; and since the 
ability to do the bard task does not so often as in drawing presuppose ability 
with (or quick acquisition of) the easy tasks. In typewriting, speed is ob- 
viously relatively more important than it is in drawing or spelling. This is 
partly because the typist early learns to do all or nearly all of the tasks, and 
thereafter improves his ability by learning to do them without lapses into 
error, and more quickly. 
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may he ascertained ivith a high degree of probability by 
submitting the individual or the group to many graded 
series of tasks, each series being made with the intent to 
have tasks spaced at equal intervals of difficulty and to 
have as many tasks at any one level of difficulty as at any 
other, by individuals or committees uninfluenced by any 
preconceptions about the form of distribution. The score 
to be given is a level score, like those derived and used in 
Chapters IX, XI, and XIII. Whatever distribution is ap- 
proximated by the average of these distributions, and more 
closely when the scores for two or more are averaged than 
when they are used singly, has a strong probability of being 
near to the form of distribution which the altitude of that 
ability would show if measured in truly equal units. 

There may be other useful ways of estimating these 
amounts of difference. In the case of the beauty of draw- 
ings, a consensus of opinion may be preferable, since 
amount of beauty may be precisely the amount of beauty 
as felt. In the case of ability to discriminate pitch, it may 
be permissible to forego equality in units of the ability 
itself and be content with the mere definition of the levels 
in terms of the physical differences; or the progress of 
research may justify some method of inference such as that 
amounts of the ability are invei*sely proportional to the 
time required to perceive the difference. 

10. Altitude of intellect is distributed approximately in 
Form A in the case of grade populations from Grade 6 to 
Grade 13, and of age populations from 11 to 14. There 
is a substantial likelihood that any one of these grade popu- 
lations will show a similar form of distribution in the case 
of the altitude of academic abilities (like ability in reading, 
or in the use of language, or in academic information) 
which are bases for promotion and are rather closely corre- 
lated with intellect. There is some likelihood that age popu- 
lations will show a similar form of distribution for altitude 
of general sensory acuity, or for general sensori-motor 
capacity, or for general motor skill, or for general mental 
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health, since variations in these amongst childi, 
age are probably in large measure due to the san^* 
causes as ‘are variations in intellect in children ot 

11. An ability is defined by making a series’® 
such that the score in this total series depends tasks 

ity. Thus an aJbility is defined by a total series ' • 

This series may be constituted by assuming that tdsks. 
such tasks are as a whole a measure of the ability, ♦ 
suming that the ranking of individuals in an order ®'f’ 

amounts of the ability by some defined consensus o ^ ^ their 
is valid, and selecting a series of tasks success sj: 
correlate perfectly with that order. This is in 
bitrary. If the ability does correspond to some ^ 
coherent, fundamental fact in the world, later wo>. 
the scries of tasks will, to some extent at least, rev with 
such is the case. The correlations of parts of th.if 
with other parts will be high ; their correlations wi ®®ries 
ties outside the series will be lower; the relaiions to’ 
ity, to age, and to sex will be fairly simple. 
able hypothesis concerning the physiological ^ )arallel‘^^”' 
ability is likely to suggest itself. If the abilit/«2>y doe 
correspond, but is just a name for some com flcateni**"^ 
aspects which various fundamental abilitk lon of 

rather artificial conditions, later work wii ofh ^ under 
tasks will probably show it. senes of 

12. Tho abmty having been defined b;j^ a tot^ 
tasks, zt may be arranged in subseries o composite t h 
in an approximate order of difficulty by ^^t'^periment 
sensus or both. con- 


13. Each composite task should be repr^^^sentative of 
total series qualitatively and should he larges, ^ 

fa, (letting a stand for the total ability) is ® or a^ 
as the self-correlation of the composite per^^2,its 

14. Composite tasks will be efficient i^nstruments fnr 
measurement in proportion as their sing% eleme t ^ 
equal in difficulty ( not necessarily in speJcific difficult 

the ability in question) and give a high rfnultiple correlation 
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15. These composite tasks should he measured in re- 
spect of their differences in difficulty (for ability a) one 
from another. Even a very inadequate measurement is 
better than none, so long as its inadequacy is nowise con- 
cealed. For we have everything that we had before the 
measurement was made, undisturbed by it. For example, 
suppose that in motor skill, composite tasks a, b, c, d, e 
. . . h are devised and tested with populations at age 2, 
4, 6, 8, 10, 12, 14, 16, and 18. Even if we do not know the 
forms of distribution of altitude of motor skill in these age 
groups at all accurately, it is better to make a reasonable 
h3q)othesis about them and act on it than to remain con- 
tent with the mere rank order for motor difficulty, that 
a<b<c<d, etc. Whenever anyone does anything with that 
rank order in the way of using it in the comparison of any 
difference or the expression of any relation, he makes some 
assumption about the amounts of difference b-a, c-b, d-c, 
etc.® There is no workable arithmetic of pure ranks, and 
cannot be. So long as it is unknown whether c-b is equal to 
b-a, or ten times b-a or one-tenth of b-a, and the like, we 
cannot add, subtract, multiply, or divide with rank differ- 
ences as such. 

16. These composite tasks should also he measured in 
respect of their differences from an approximate absolute 
zero of ability a. Even though the location of zero is hypo- 
thetical and has a large margin of possible error, the esti- 
mate can do no harm if its unreliability is not concealed. 
Nothing is distorted by it. It is simply an added feature 
to be used or neglected as seems wise. Anybody who 

0 Thus the Spearman correlation formula 
T = 2 COB g (1 - E) - 1 where E = 1 - 

assumes that the form of distribution of the two traits is Form A, that of the 
normal probability surface. 

Spearman formula, r = 2 sin (g p) 

n(n* — 1) 

where p = l--g255— 

assumes that the form of distribution of the two traits is Form A. If the 
form is rectangular, r is taken as equal to P. 
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wishes to utilize the “times as mnch*’ comparison wdth 
the facts will have to make some assumption about the loca- 
tion of zero ; and the assumption made by the author of the 
scale is likely to be better than the average of the assump- 
tions of those who use it. He has presumably more knowl- 
edge and better judgment about it than the average of them. 
No one of them is compelled to accept the location which 
he assigns. 

A qualification of “usually’^ might have been inserted 
in the paragraph above, because there may be important 
abilities where any location of the absolute zero is so diffi- 
cult that any attempt to do so with present knowledge is 
pretentious and absurd. We doubt this, however. On the 
contrary, when an ability has been defined by a series of 
tasks, the extension down from the easiest composite task 
to a task of the same general sort which requires just a 
bare trifle of the ability in question is almost a necessary 
consequence. If a reasonable location of the absolute zero 
cannot be attained, it is a sign that the ability itself is not 
an important, unified variable in nature. 

There are two important special cases of locating abso- 
lute zero which may suitably be considered here. The first 
is that exemplified by discrimination of pitch, where it is 
customary to measure down from an upper limit of perfec- 
tion, using the amount of error made, or the smallest 
amount of difference discernible, and thinking of its dif- 
ference down from zero error rather than up from zero 
ability. This has its advantages, and it need not be alto- 
gether discarded ; but it pi'oduces numbers which are very 
ill adapted for quantitative thinking, and may well be sup- 
plemented by measures up from a zero, meaning just barely 
not any ability. 

The second is the case where the standard point of refer- 
ence is the ordinary or modal man. Thus, people com- 
monly measure popularity up and unpopularity down from 
a condition of average popularity. This procedure has 
been widely recommended in recent psychological and edu- 
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cational science, and has the very great merits that its 
point of reference is one near where the measurements are 
made, is easily definable in reality, and is convenient for 
studies by correlation and partial correlations. Measures 
+ and — from such a central tendency in units of the varia- 
bility of the group are also honest, making no pretence at 
being more than they are. It has the disadvantage of in- 
stability, shifting with the group taken. Again science 
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may retain all these advantages, and still add the different 
advantages of a true zero. 

Finally, attention may be invited to the difference be- 
tween natural and conventional zero points. In the former, 
there is a genuine beginning of an important natural phe- 
nomenon. In the latter, there is a beginning only from the 
point of view of some human institution or custom. Thus 
motor skill in the case of the hand may be said to have its 
task of zero difficulty somewhere below the point of reach- 
ing toward an object and touching it, or of grasping an 
object touched and bringing it somewhere near the month. 
This is near a natural zero. The tasks of zero difficulty 

33 
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for ability in penmanship and of zero difi&ciilty for ability 
in drawing would by any reasonable view be tasks which 
would be much above this zero of manual motor skilL The 
zero task for penmanship would be to make something 
which could be identified as writing, though no letter or 
word could be identified. For example, to copy Fig. 70 
with the result shown in Figure 71 may be regarded as 
near the zero of handwriting. To use pencil or crayon to 
make a line as “good’^ as Figure 73 when Fig. 72 is 
shown as the model may be regarded as near the zero of 
drawing. To use pencil or crayon to make any kind of a 
scribble such as Fig. 74 or Fig. 75 may be regarded as zero 
for the use of a mark-making tool. All three of these zeros 
are well above the zero of manual motor skill. They come 
later in life’^ and are impossible for idiots for whom the 
reaching and grasping are possible. 

The number of conventional zeros is legion. We can 
have tasks of zero difficulty for ability in typewriting, in 
stringing beads, in shuffling cards, in playing the piano, in 
playing the violin, in tapping, and scores of other varieties 
of manual skills ; and for certain educational and economic 
problems (‘ach of these zeros may be of value and the refer- 
ence of measures of ability to it may be desirable. 

They will be even more useful if each of them is mea- 
sured off from the natural zero for manual motor skill, so 
that reference can be made to it also. This holds as a gen- 
eral iirinciple. Any conventional zero will be made more 
useful by being itself measured off from some natural zero. 

There are two important fields of mental measurement 
where the principles and techniques which we have devel- 
oped for intellect do not apply, at least not without radical 
changes. 

The first is the general case of the description and mea- 
surement of the connections or bonds whereby any given 

^ The dates may be set roughly as i yr. for reaching and grasping, 1 year 
for making a mark, and 2 years for copying a line. The date for malcit.g 
scything enough distinguishable from general scribbling and mark^making to 
be called a zero specific to handwriting ability is not known, but is later 2 
years. 
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condition or state of affairs in a mind leads to another con- 
dition or state of affairs in that mind. In such cases we 
need not and do not nse valuation or the concept of diffi- 
culty, but simply record that such and such connections exist 
in such and such degrees of strength from zero up. By 
the strength of a connection between condition A and con- 
dition B, we mean the probability that A will be followed 
by B. The strength of the connection is measured crudely 
by the percent which A-^B is of A— A->C, 4* A-»I), 
. . . -f- A N,® and by the length of time that the connec- 
tion will last without exercise ; and somewhat more exactly 
by this percent and this time in relation to the number and 
strength of competing tendencies. This is by far the com- 
monest sort of mental measurement. 

The second is the measurement of likes and dislikes, in- 
terests, desires, ‘ ^drives, motives, or whatever one chooses 
to call the facts whereby certain states of affairs are satis- 
fying and others are annoying to the animal. The whole 
subject of satisfiers and annoyers is in more or less doubt 
and dispute, but by any view the measurement of how 
much a man likes the taste of olives or a brisk run up a 
hill is vei-y different from the measurement of either his 
intellect or his tendency to contract the pupil in a bright 
light. 

The most striking difference is that the measurement 
is now up and down from a zero of indifference to A, where 
the animal neither seems moved to do anything which will 
change the state of affairs in respect of A, nor seems moved 
to do anything which will cause A to continue or recur. 

Another notable fact is that whereas it is easy to dis- 
cover whether A or B is more satisfying to any given in- 
dividual X, it is hard to discover whether A is more satis- 
fying to individual X than to individual Y. 

According to our notions, the satisfyingness or annoy- 
ingness of any state of affairs to any mind is measurable 

8 The cases where A occurs and apparently leads to nothing are to be in- 
cluded in the count. 
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by the products prodnced, including the internal neural 
products which lead to honest reports of satisfaction and 
annoyance, welcome and rejection, ‘‘for’' and “against.” 
We also think that different varieties of satisfyingness 
such as that of food when hungry and hearing music and 
being looked at with respectful glances, are intercommen- 
surable ; and also susceptible of algebraic computation 
against amounts of annoyingness. The discussion of any 
theories of such an hedonic calculus is, however, beyond 
the scope of this chapter or this book. 



APPENDIX I. 


The Form of Distribution of an iNDiviDUAii's 
Variations in Intellect 

We have elsewhere' reported evidence showing that the 
variations of an individual in separate tests of an ability, 
each a half hour or so in length, were distributed almost or 
quite symmetrically about his average or mode, extreme 
deviations upward being almost or quite as common as ex- 
treme deviations downward. It is the purpose of this ap- 
pendix to present, in a very brief and summary manner, 
additional evidence to the same effect. 

We have confirmed the results in the case of the Army 
Alpha by using the records of 81 high school students in 
Milan, Michigan, each with three trials of Alpha, a year 
apart, given by Superintendent Tape. (Fall of 1919, 1920, 
1921.) Fifteen students were at those dates eighth grade, 
freshmen, and sophomores, respectively; thirty- two were 
freshmen, sophomores, and juniors; and thirty-four were 
sophomores, juniors and seniors. The allowances for prac- 
tice effect and growth were : first group, 15 and 13 points ; 
second group, 16 and 12 points; third group, 13 and 13 
points. (The average scores were: first group, 91, 105, 
and 121; second group, 87, 103, and 115; third group, 104, 
117, and 130.) The median of the scores thus corrected 
was found for each individual, and the two deviations from 
this median. The distribution of these plus-minus devia- 
tions is shown below. The 81 medians themselves are not 
included. 

The average deviation upward is 8.60 points. The aver- 
age deviation downward is 8.54 points. There are 23 plus 
deviations and 24 minus deviations which are 10 points or 
greater. 

1 Jour, of Ezp. Pay., voL 6, pp. 161—167. 
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Frequencies Total Grouping 

Deviations Ist Group 2nd Group 3rd Group Total by 9 Points 


-32, 

33, 

34 


1 


1 

1 

-29, 

30, 

31 



1 

1 


-26, 

27, 

28 


2 


2 

3 

-23, 

24, 

25 






-20, 

21, 

22 


3 


3 


-17, 

18, 

19 

1 


2 

3 

12 

-14, 

15, 

16 


2 

4 

6 


-11, 

12, 

13 


4 

3 

7 


- 8, 

9. 

10 

2 

7 

7 

16 

40 

- 5, 

6, 

7 

5 

5 

7 

17 


o 

3, 

4 

6 

3 

9 

18 


- h 

0, 

+ 1 

1 

9 

6 

16 

53 

+ 2, 

3, 

4 

5 

7 

7 

19 


+ 5. 

6, 

7 

3 

9 

5 

17 


+ 8, 

9, 

10 

3 

4 

5 

12 

35 

i 11, 

12, 

13 

1 


5 

6 


^ 14, 

15, 

16 

1 

5 

2 

8 


+ 17, 

18, 

19 

] 

1 

O 

4 

13 

+ 20, 

21, 

22 



1 

1 


+ 23, 

24, 

25 



1 

1 


+ 26, 

27, 

28 


1 


1 

4 

+ 29, 

30, 

31 

1 

1 


2 


4 32, 

33, 

31 



1 

1 

1 


39 pupils in Grades 5 and 6 took the Courtis tests in 
computation four times durinf^ the year. The effect of 
practice was allowed for hy adding to the first score the 
average gain of trial 2 over trial 1, and subtracting from 
tlie third score the average gain of trial 3 over trial 2. 
Trial 4 was treated similarly. The deviations of each in- 
dividuaPs scores from his median score were then com- 
puted. As the result, we have ; 


Deviations 

Frequencies 

- 8 to - 10.5 

1 

-5 - 7.5 

2 

-2 “ - 4.5 

19 

-1.5“ + 1.5 

107 

+ 2 “ + 4.5 

20 

+ 6 '*+ 7.5 

4 

+ 8 “+10.5 

0 
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37 pupils in Grades 5 and 6 took the Woody Test, Series 
A, three times in Oct. (or Nov.), Jan., and May of a 
school year. The scores were corrected for the practice 
effect by adding the average gain of trial 2 over trial 1 to 
the score for trial 1, and subtracting the average gain of 
trial 3 over trial 2 from the score for trial 3. The devia- 
tions of each individual’s scores from his median score 
were then computed. As the result, we have, including 
the 37 deviations of 0 belonging to the median measures : 


Deviations 
-23 to -31 
-14“ -22 

- 5“ -33 

- 4 “ 4 4 
+ .'>“4-13 
4-14“ 4-22 
4-23 “ 4-31 


Frequencies 

1 

3 

16 

83 

8 

2 

0 


In records in tests of spelling treated in a similar man- 
ner, we have, as the distribution of an individual’s devia- 
tions from his median, the following: 


Deviations 
-32 to -40 
-23 -31 

- 26 '< -22 

- -13 

- + 4 

+ 19 ‘‘ +22 
+ 23 ‘‘ +31 
+ 32‘‘ +40 


Frequencies 

0 

2 

5 

24 

53 

30 

4 

1 

1 


Pupils in Grades 4, 5, and 6 were tested with Stanford 
Binet, National A, National B, Otis Advanced,® Myers Men- 
tal Measure, Haggerty Delta 2, Illinois, and certain parts 
of Dearborn. Each score was first turned into a devia- 
tion from the median for the group in that test, in terms 
of the variability of the group in the test in question. Then 

2 Some had the Otis Primary instead. For these, estimated scores in the 
Otia Advanced were computed. 
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it was expressed as a deviation from the average of the 
eight such deviation-scores for the individual in question. 
These last deviations represent the variability of an in- 
dividual around his average ability in intelligence tests. 
Their distribution is as follows : 


Deviations 
- 165 to - 194 
-135*^ -164 
-105 -134 

- 75*^-104 

- 45 - 74 

- 15'* - 44 

- 14 "-1- 14 
-h 15"+ 44 
+ 45" + 74 
+ 75" +104 
+ 105" +134 
+ 135" +164 
+ 165 " +194 


Frequencies 

0 

2 

1 

9 

40 
124 
137 

86 

41 
10 

3 

2 

1 


65 pupils in Grades 8 to 12 were tested with Alpha Form 
5, Alpha Form 8, Terman Group Test Form A, Terman 
Group Test Form B, and half of Part I of the Thorndike 
Examination for High School Graduates. These five scores 
for each pupil were treated just as the eight scores de- 
scribed in the previous paragraph, except that the final 
deviations are deviations from the individual’s median in- 
stead of from his average. 

The resulting distribution was as follows (including the 
65 zero deviations of the medians themselves) : 


Deviations 

Frequencies 

-no to 

-129 

1 

- 90 “ 

-109 

2 

1 

O 

- 89 

10 

- 50 

- 69 

10 

o 

eo 

1 

- 49 

34 

- 10 

- 29 

47 

- 9 ** 

+ 9 

117 

+ 10 

+ 29 

55 

+ 30 

+ 49 

27 

+ 50 ** 

+ 69 

15 

+ 70 

+ 89 

5 

+ 90 ** 

+ 109 

2 

+ 110 

+ 129 

0 
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The variations in I. Q/s from different Stanford Binet 
examinations at long intervals are not exactly the sort of 
evidence that we wish, since variability is complicated with 
growth and the measure is an indirect ratio, but it is of 
interest to note the facts: So we have tabulated the devia- 
tions of an individual’s separate I. Q.’s from his median 
I. Q. in all of the cases of Baldwin and Stecher ( ’22, p. 24) 
who had at least three separate examinations. The results 
are as follows: 


Deviatioiis 


Frequencies 


-26i to -29i 1 

-23 ** -26 1 

-19i “ -22i 3 

-16 -19 5 

-12i -15i 6 

- 9 ** -12 19 

- - 81 28 

- 2 - 6 53 

- li + 11 149 

+ 2 “ + 51 60 

+ 51 + 81 27 

+ 9 “ +12 14 

+ 121 “ +151 5 

+ 16 “ +19 1 

+ 191 “ +221 2 

+ 23 “ +26 1 

+ 261 “ +291 0 


In no one of these six cases is there more than a very 
slight excess of extreme downward deviations. On the 
whole, the balance is about even as shown by the two 
rough summaries below. 

We may expect, in a thousand deviations, about 12 that 
are 2%o or more minus to about 7 that are 2%a or more 
plus, if we assume that the scoring units in the cases studied 
are on the average of about the same real value at the lower 
as at the upper extreme of the ranges studied. About the 
real value of the units we know very little, but it seems 
likely that the tests have more tasks at the lower levels of 
difficulty tlian at the upper, rather than the reverse. So 
getting equal units would probably reduce the excess of 
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Courtis 

+ 

Woody 


+ 

spelling 

Total 

low 

0 

+ 

0 


+ 

0 

0 


1 

+ 

1 



2 

4 


2 

+ 

3 


+ 

5 

10 


19 

+ 

16 


+ 

24 

59 

central 

107 

+ 

83 


+ 

53 

243 


20 

+ 

8 


+ 

30 

58 


4 

+ 

2 


+ 

4 

10 


0 

+ 

0 


+ 

1 

1 

high 

0 

+ 

0 


+ 

1 

1 


8 test + 


5 test 

+ 

repeated I.Q. 

Total 

low ] 

t 0 + 


0 



1 

1 


\ 2 + 


3 

+ 


9 

14 

1 

50 + 


54 

+ 


53 

157 

central 

- 347 + 


219 

+ 


262 

828 

J 

54 + 


47 

+ 


46 

147 

high ] 

i 3 + 


2 

+ 


4 

9 

J 

[ 0 + 


0 



0 

0 

low deviations rather than increase it. 

Also, the excess of 

low deviations will be 

reduced 

if 

Wil 

! omit the 

somewhat 

ambiguous I. Q. data. 







These facts with those of the previous report certainly 

justify the 

conclusion 

that the 

real 

variations 

of an in- 


dividual in a 30-minute test of intellect or school achieve- 
ment 'will be found to be approximately symmetrical. This 
justifies the use of the avera^?e in practice, and clears the 
way for important advances in the theory of mental mea- 
surements and the scaling of tasks. 



APPENDIX 11. 


The Relation op an -Individual's Variability to His 
Ability in Tests of Intelligence 

An individual who takes a number of different trials in 
a test, using alternative forms arranged to be of equal diffi- 
culty, attains varying scores. It is of importance for many 
purposes to know how the variability of an individual in 
any ability is related to the amount of the ability which he 
has, that is, to his average or modal achievement. We 
have investigated the matter somewhat thoroughly in the 
case of seventeen tests of intelligence. 

It is necessary to distinguish between (a) the apparent, 
or face-value, relation observed between the variability of 
an individual's separate scores and his average score, and 
(b) the real relation that would be observed if these scores 
were transmuted into measures such that 1, 2, 3, 4, 5, 6, 7, 
etc., represented a real arithmetical progression of amounts 
of the ability- For example, we find that twenty individ- 
uals each of whom took (after two preliminary trials, to 
eliminate the practice effect) from eleven to thirteen forms 
of Part I of the Thorndike Intelligence Examination for 
High School Graduates, showed the results of Table 138. 
If the scores are taken at their face valae, it appears that 
the variability of an individual whose median score is about 
105 (from 100 to 113) is very nearly the same as the varia- 
bility of an individual whose median score is about 128 
(125 to 132). If, however, the units of the scoring from 
90 to 120 really represent smaller increments of ability than 
the units from 120 to 145, the real variability of an in- 
dividual of ability 105 is less than the real variability of 
an individual of ability 128. The converse is true, if the 
units of the scoring scale from 90 to 120 really represent 
larger increments of ability than the units from 120 to 145. 
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The facts concerning the scores taken at their face value 
are worth knowing and recording, but it is the relation of 
the *‘rear’ variability to the ‘‘rear’ amount of ability that 
is the essential problem. Our procedure and reasoning are 
as follows: 

We record the face-value score results for many differ- 
ent sorts of tests of the ability in question, and note in each 
case any facts about the construction of the tests which 
concern the probability that its units progressively swell or 
shrink in “real” value over any considerable fraction of 
the range we are concerned with. We note especially the 
score results in those cases where there is no reason to 
expect swelling more than shrinking. The average rela- 
tion between variability and ability found in these cases 
may be taken to represent approximately the real relation, 
until someone produces evidence that, in all or nearly all 
tests for the ability in question, there are forces leading 
psychologists, quite without intention, to devise scoring 
plans which make for progressive swelling or shrinking of 
units at the same places along the scale of real ability. 

The most desirable material for our purpose would be 
the records of individuals representing a very wide range 
of ability, there being many of them, and each being mea- 
sured with many alternative forms of the test in question. 
Kange is desirable to accentuate the relation and measure 
it throughout. Large populations are desirable to reduce 
the disturbing effect of individual differences. A large 
number of trials of the test is desirable to locate exactly 
the ability to which the variability is related. Unfortu- 
nately, there is no such material, and we have to do the 
best we can by putting together several small ranges, and 
by using individuals who have been measured by only two 
trials of the test. Indeed, we use individuals who have 
been measured by only one trial, by a method which will 
be described shortly. Whatever the method, however, the 
result is an estimate of what the relation would be between 
(o) the variability of an individual in an infinite number 
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of trials or tests or measures of the ability and (b) the 
amount of his ability as measured by the mode or other 
central tendency of these separate trials or measures. 

In general, any material is useful for our purpose by 
which we can measure the variability of a number of trials 
or individuals w^hose central tendency in the ability is low, 
and also the variability of a number of trials or individuals 
whose central tendency in the ability is high. The essen- 
tial difference is that in certain material the relation is 
very much more liable to blurring and distortion by the 
inaccuracy of the determinations of average ability itself 
than it is in other material. 

The effect of the inaccuracy of the averages upon our 
determinations of the relation between average intellect 
and variation in intellect is somewhat subtle and compli- 
cated. We need not discuss it here. After we have gath- 
ered and organized the best data we can bearing on the 
relation, and found, as we shall, that they are best ex- 
plained by the hypothesis that the variability of an in- 
dividual neither increases nor decreases according as his 
average is low or high, we shall have to ask whether the 
case would be otherwise if the inaccuracies of these aver- 
ages were reduced. 

When we have only two separate trials with a test, it is 
best to measure an individual’s variability by the difference 
between the two scores, after suitable correction for the 
practice effect and for differences in the difficulty of the 
forms used. (Call this difference D.) The average varia- 
tion of an individual from his average or median (call this 
A.D.) may be taken as this difference divided by V2. The 
average of the A.D.’s of a group of individuals, all of the 
same or nearly the same average ability, is the average of 
their D’s divided by ■\/2. Table 139 shows the results in 
the case of a set of scores in the Thorndike Part I, series 
of 1919 to 1922, so treated. 

If we obtained the actual average for each individual 
and computed his A.D. directly by the formula, Z (devia- 
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tions)/2, we should have A.D.*s throughout seven-tenths as 
large as those of Table 139. This would be because in each 
case the central tendency had been automatically chosen so 
as to vary from the true central tendency in the direction 
of making the A. D. a minimum. 

A still more convenient way at times is to estimate the 
variability of an individuaPs separate trials around his 
average indirectly from the variability of n individuals 
(each measured by a single trial) around the average of 
the n individuals, using individuals all equal in ability to 
the individual in question. If n individuals in one trial 
with Army Alpha all score 165, and in another trial score 
168, 160, 171, 159, etc., etc., averaging 165, it may be argued 
that the deviations of the 168, 160, 171, 159, etc. from 165 
are comparable and proportional to the deviations of one 
individual of ability 165 who took Alpha n times.® More 
exactly, the facts are that any one of n individuals who 
are all really of ability K in a test does resemble any other 
of them in variability ; and that if there is taken from each 
of the n distributions representing their separate variabili- 
ties, one variation at random, the composite of these se- 
lected variations will resemble the distribution of the varia- 
tions of any one of the n individuals. In the particular 
matter with which we are concerned, the average of these 
selected variations will vary only by chance from the aver- 
age of the averages of the separate individuals’ variations, 
and this will vary only by chance from any one of these 
averages. The matter of concern to us is that, when n in- 
dividuals are measured each by only one trial and all score 
K, they reaUy are not all of ability K, but vary around it. 
Consequently, when n individuals measured each by one 
trial are used in place of one individual measured by n 
trials, the attenuation” or blurring of the relation is much 
greater. Using one individual measured by n trials relates 
the variability to one precisely located amount of ability. 

* Assuming of course that in both cases the effects of practice, interest, 
and the like are properly allowed for. 
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Using one trial of each of n similar individnals relates it to 
an amount which is located with a considerable error. If 
we keep this attenuation in mind and allow for it, we may 
use material of this sort when it is desirable. 

Even if we have only one trial with a test, it is still 
possible to estimate the approximate average of the aver- 
age variations of a group of individuals each from his 
average, provided the individuals are also measured by 
some test which correlates fairly closely with the test in 
question. For example, suppose that ten individuals all 
score 75 in the score for one trial of the entire Thorndike 
Intelligence Examination for High School graduates, and 
score 30, 32, 34, 35, 36, 37, 38, 40, 42, and 48 in Part III 
of it; and that the general correlation between a one trial 
score for the entire examination and average ability in a 
large number of trials of Part III is .90 ; and that the corre- 
lation between the average score in two trials of Part III 
and the average ability in a large number of trials of Part 
III is .90 for a similar group. Then we do as well to take 
the variations in Part III of the ten individuals who all 
scored 75 in the entire examination as we would do to take 
the variations of ten individuals who all scored alike in the 
average of two trials with Part III itself. 

We shall make very large use of this indirect method 
because by its use we can cover wide ranges and have large 
populations, and also in certain instances because the ma- 
terial available can be treated by no better method. 

It should be noted that when the two trials of a test by 
an individual occur in a single session, the variability mea- 
sured is that due to the difference of one hour from an ad- 
jacent hour, plus, of course, the differences of the content 
of the two forms of the test that were used. Cases of 
single-session variability will be so designated wherever 
they appear in what follows. Except for them, the varia- 
bility studied is that of a random sampling of different days 
of an individual (but omitting days when by sickness or 
other causes he would not be submitted to a test), with a 

34 
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Tandom sampling of alternative forms of the test in ques- 
tion. 

Let ns first illustrate certain further matters concern- 
ing method. Table 138 has already shown the method 
where we have several trials with the test by an individual 
on different days. Table 139 has shown the method where 
we have two trials by an individual.* Table 140 below 
shows the method where we use the array of scores in trial 
2 of n individuals scoring alike in trial 1 to represent ap- 
proximately the n different trials of one individual. The 
test is still Part I of the Thorndike Examination, as in 
Tables 138 and 139. 

It will be noted that in computing the variability of an 
array we use as a central tendency, not the actually ob- 
served average or median of that array, but the central ten- 
dency which seems most probably the true one for it in 
view of the facts of the entire table, that is, the central 
tendency given by the total relation line or regression. If 
the actually observed central tendencies were used, there 
would result a fallacious diminution of the variability in 
all arrays, and this would sometimes be a very large error 
when the number of cases in the array was small.® The re- 
lation line is determined by plotting the observed medians 
and drawing a smooth line such as approximately (1) 
makes the sum of the plus deviations equal to the sum of 
the minus deviations from it, and (2) distributes these 
deviations about equally + and — along the course of the 
line, and (3) has d geometrical form not much variant from 
the form found in general for the test in question. (That 
form is, for most of our material, rectilinear). There is 
thus an element of personal judgment in the decision as to 
the relation line and consequent placement of the central 

* Theae were on the aame daj in immediate sequence giving single^session 
variability* 

^ For example, the three cases in Table 140 arrayed under 45 would show 
an average deviation of 4.5 from the average obtained by considering them 
alone, whereas they show an average deviation of 8.35 from the average that 
the whole table indicates as the probably true average for that array. 
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tendencies from which the variabilities were computed. 
This, in general, has almost no effect save upon the arrays 
toward either end, and its effect is greatly reduced so far 
as our general problem is concerned, by reason of the over- 
lapping of the groups which we have used. Finally, the 
personal judgment did not, so far as we know, favor any 
one sort of relation of variability to magnitude more than 
another ; it acted as a chance error. 

The midpoint of a class interval of the array which was 
nearest to the relation line was used as the central tendency 
from which to compute the variability of the array. The 
reason for adopting this crude procedure instead of com- 
puting the lines of best fit, and computing the variability of 
each array from the exact point on it that belonged to the 
array in question, was that we had to choose between 
spending our time on a few relations computed exactly and 
on many relations estimated in this rough way. The latter 
procedure seemed much more instructive per unit of time 
spent. 

Table 141 shows the method where we use the array of 
scores in one trial in Test X of n individuals scoring alike 
in some other test closely enough correlated with Test X 
to serve instead of a trial with Test X itself. The central 
tendency of an array from which to compute its variability 
was determined just as in the case of Table 140. 

We have four tables of the type of Table 139, and six- 
teen of the type of Tables 140 and 141, for Part I of the 
Thorndike Examination. Their summarized results ap- 
pear as Tables 142 and 143. Our next task is to combine 
them to give a general estimate of the relation of the varia- 
bility of an individual to liis ability in the case of this test. 
There are many suitable ways of doing this, each having 
certain merits. The method which we have adopted is to 
express each entry of a line of Table 142 as a percent of 
that line’s average® entry under abilities 70 to 89 and to 
average the percents under each ability after weighting 
them as follows : 

0 This is a rou^^hlj weighted average. 
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N Weight 

1 or 2 1 

3-5 2 

6-12 3 

13-19 4 

20-29 5 

30-39 6 

40-59 7 

60-79 8 

80-99 9 

100 — over 10 

The use of the percents makes it convenient to compare 
the relation found for this test with that found for others. 
The particular weighting used gives some weight to each 
group studied apart from the number of cases it contains, 
and adds no weight to size of population over 100 in any 
one array in any one group, and economizes time. The 
same procedure was followed with the facts of Table 143. 

The relationship shown by the tinal line of Table 142 
and the final line of Table 143 may be seen more clearly 
if we group the results more coarsely. In combining for 
the coarse grouping, the average percentages already ob- 
tained are weighted each by the sum of the weights at- 
tached to it. Thus 133 with a weight of 5 and 50 with a 
weight of 2 become 109 with a weight of 7 ; 94 with a weight 
of 6 and 79 with a weight of 3 become 88 with a weight 
of 8; 116 with a weight of 9 and 90 with a weight of 8 
become 104 with a weight of 17. The relations of Tables 
142B and 143 with coarser grouping are shown in Table 
144. It should be observed that the grouping carried out 
in this manner after the variability of groups or arrays 
has been obtained from the original fine grouping, does 
not add any new ^‘attenuation.’* All groupings in what 
follows were carried out in this manner. 

We have, in the case of many tests, material of the sort 
shown by Tables 142, 143, and 144 for the Thorndike, 
Part I. In two instances results of different tests have 
been combined to abbreviate the presentation. The first is 
where results of the regular Army Beta and the weighted 



TABLE 140. 

Thornsiki! Ikt. Exak. for H. S. Oeaduates, ’19- *22 Series. Part I, Trial 2, Arrated under Trial 1. Test of Per. '22. 30=30 to 34, 

35 = 35 TO 39, ETC. 
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TABLE 141. 

Thobnmkb Int. Ex. toe H. S. Qbadxtatm, *19- *22 Sbehs. Pakt I, Teial 2, Abrayed tjmdeb Thorndike Exam., Total Score. Women Stu- 
dents IN High School, Normal School, College and Universitt. 30 = 30-34; 35 = 35-39, etc. 
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TABLE 142(A). 

Thorndhcb Int. Ez. for H. S. Orabuatrs. Part I Average Difference between Two Trials 

_ « _ _ / Sum OP Trial 1 AND Trial 2 \ 

()91N01jE SiSUBIUN) in JWSliATlUN TO TJIE AVEttAuE OOOAIQ ( q /' 


A = Nonnal School students. C = Night school men. 

B = Candidates for college entrance. D = S. A. T. 0. candidates. 

(The average here is of 3 trials. The difference is Trial 3 -Trial 2.) 


Average score 15 

20 

25 

30 35 40 

45 

50 

55 

60 

65 

70 

75 

80 

Average 

Difference A 






10.5 

12.0 

9.0 

8.3 

9.3 

8.0 

10.1 

7.9 

n 






6 

12 

9 

34 

48 

61 

84 

103 

Average 

Difference B 








9.0 

10.0 

13.0 

6.8 

10.0 

9.0 

n 








2 

3 

3 

12 

12 

10 

Average 

Difference C 

9.75 

10.5 

9.0 

10.0 

6.0 10.2 

8.1 

3.8 

6.0 

9.8 

8.1 

8.0 

10.1 

7.0 

n 

4 

2 

4 

6 

7 10 

14 

18 

12 

22 

26 

22 

16 

24 

Average 

Difference D 










11.5 

4.9 


9.4 

n 










6 

22 


22 

Weighted Average 
Percent of the Va- 
riability at 70-89 

117 

127 

108 

120 

72 123 

107 

84 

90 

108 

117 

84 

118 

100 

Sum of weights 

2 

1 

2 

3 

3 3 

7 

7 

8 

13 

17 

21 

16 

23 

Average score 

85 

90 

95 

100 

105 

no 

115 

120 

125 

130 

135 

140 

145 

Average 

Difference A 

9.2 

8.1 

8.4 

7.8 

8.1 

7.8 

6.7 

4.5 

6.7 

6.0 

8.3 



n J45 

145 

115 

122 

102 

65 

50 

20 

13 

3 

4 



Average 

Difference B 

7.6 

7.5 

8.5 

7.9 

7.6 

6.2 

7.1 

6.6 

6.8 

5.4 

7.4 

4.2 

15.0 

CO 

30 

32 

42 

49 

52 

65 

43 

37 

19 

la 

5 

1 

Average 

Difference C 

8.8 

5.0 

7.1 

5.7 

7.5 

6.3 

7.5 

10.5 

10.5 

4.5 




n 18 

20 

17 

9 

10 

9 

2 

2 

3 

2 




Average 

Difference D 

8.4 

7.6 


6.1 

6.0 

4.9 


6.5 

5.8 

4.4 




n 48 

63 


113 

120 128 

114 

70 

23 • 




Weighted Average 
Percent of the Va- 














riabiUtyat 70-80 103 

88 

96 

85 

87 

75 

82 

78 

76 

62 

92 

52 

185 

Sum of Weights 

27 

29 

20 

30 

30 

28 

16 

23 

20 

12 

6 

2 

1 
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TABLE 142(B). 

8i.MX AS Table 142(A), except that the difference is between Trials on Different Dats, 

AND THAT THE AVERAGE ScOBS IB FBOU FODR TRIALS. NORMAL SCHOOL STUDENTS. 


Average 

Score 15 

20 

25 

30 35 

40 

45 

50 

55 

60 

65 

70 

75 

80 

Average 

Difference 










8.0 

5.2 

10.8 

n 










2 

10 

5 

Weighted Average 
Percent of the Va- 
riability at 70-89 










102 

66 

137 

Weight 










1 

3 

2 

Average 

Score 85 

90 

95 

100 

105 

110 

115 

120 

125 

130 

135 

140 

145 

Average 

Difference 8.5 

6.3 

8.0 

7.3 

8.4 

7.4 

6.8 

6.1 

1.0 

9.0 




n 19 

21 

17 

24 

28 

18 

16 

7 

1 

3 




Weighted Average 
Percent of the Va- 













riability at 70-80 108 

80 

102 

93 

107 

94 

87 

78 

13 

115 




Weight 3 

5 

4 

5 

5 

4 

4 

3 

1 

2 





Army Beta and a certain picture test^ have been combined ; 
the second is where the results for the Haf^gerty Delta 2, 
the Myers Test, the Kelley Trabue Completion, and the 
Thorndike Visual Vocabulary are combined. In these com- 
binations each is treated separately up to the point of the 
Weighted Average Percents of the Variability at Ability X, 
and care is taken to chose Ability X to represent closely the 
same percentile ability in each of the tests to be combined. 
Consequently, two variabilities are combined, only if they 
belong to approximately the same ability. We use the 
same method of weighting that is used for any one test 
given to different groups. 

» Consisting of tests 3, 4 and 5 of the Thorndike, Part n. 



Average Deviation Times 2. 
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TABLE 143. 

Thorndik® Int. Ezau. for H. S. Graduates. Part I. Variability of Score nr One Tbiaii 
Arrayisd under Score in Another T rtat. or under Total Score in the Entire 
Examination. 10 = 10 to 14 ; 15 = 15 to 19, eto. 

A, B, C = Nig^ht-scliool men. D = College entrance candidates. E, P, G, H = Collie freshnieii 
plus some others. I, J = Women in High School, Normal School, and College. K, £ = Men of* 
college grade and below. M,N=: Summer School students at a Normal School. 

O, P = Men in High School and College. 


Central Tendency of the Array. 



10 

15 

20 

25 

30 

35 

40 

45 

50 

55 

60 

65 

70 

76 

Group A 

12.5 


30.0 


22.0 


19.0 


15.2 


16.C 


15.6 


n 

4 


4 


6 


16 


27 


35 


60 


Group R 

5.0 

6.0 

10.0 

15.0 

13.8 

8.4 

21.6 

16.8 

20.6 

16.0 

20.0 

11.2 

15.2 

14.0 

n 

2 

2 

4 

2 

8 

6 

13 

19 

14 

15 

18 

17 

19 

10 

Group C 

38.4 

10.0 

0 

10.0 

20.0 

12.0 

12.6 

10.0 

16.4 

22.5 

17.0 

14.4 

17.2 

9.4 

JL 

4 

2 

1 

4 

6 

5 

8 

7 

11 

12 

13 

18 

18 

16 

Group D 














6.0 

n 














2 

Group E 













15.0 

11.8 

n 













2 

11 

Group P 











30.0 

0 

13.0 

10.0 

n 











2 

1 

10 

10 

Group G 









10.0 


17.5 


14.1 


n 









2 


8 


27 


Group H 











20.0 


7.8 


n 











2 


9 


Group I 







40.0 

20.0 

11.7 

25.0 

30.0 

19.6 

12.5 

12.5 

n 







1 

2 

6 

2 

8 

26 

40 

64 

Group J 









20.0 

30.0 

5.0 

10.0 

15.0 

15.4 

n 









1 

2 

6 

2 

8 

26 

Group K 









20.0 


10.0 

15.0 

10.0 

18.8 

n 









1 


1 

6 

1 

8 

Group Ii 






10.0 


15.0 

10.0 

18.0 

10.0 

12.9 

10.8 

15.8 

n 






1 


2 

2 

5 

1 

7 

12 

31 

Group M 







20.0 

10.0 

20.0 

0 

17.1 

17.5 

12.8 

15.1 

n 







1 

1 

3 

1 

14 

20 

18 

71 

Group N 






10.0 

25.0 

24.0 

17.5 

13.8 

18.2 

15.6 

16.5 

17.9 

n 






1 

2 

5 

12 

16 

22 

89 

49 

67 

Group 0 






40.0 

5.0 

11.4 

14.0 

19.5 

15.0 


15.9 

14.7 

n 






2 

2 

7 

10 

21 

36 


51 

71 

Group P 








30.0 

10.0 

20.0 

18.6 

18.2 

19.0 

14.4 

n 








1 

1 

2 

7 

11 

21 

se 


Weighted Aver- 
age Percents of 
the Variabilitj 

at 70 to 89 133 50 94 79 116 90 125 106 101 120 114^ 99 96 96 

Sum of Weights 5 2 5 3 9 8 15 16 28 22 45 64 66 
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TABLE 143 — {C<mtimbed) 




80 

85 

90 

95 

100 

105 

110 

115 

120 

125 

130 

135 

140 

145 


Group A 

!2.7 

18.3 


14.0 


13.3 


15.0 


10.0 


20.0 




Q 

37 

41 


30 


18 


6 


4 


2 




Group B 

12.9 

11.8 

13.0 

15.6 

5.0 

16.6 

16.6 

10.0 








u 

7 

11 

10 

9 

4 

3 

6 

2 








Group C 

16.7 

18.8 

16.4 

7.2 

12.0 

10.0 


15.6 

10.0 

10.0 






jr 

15 

16 

11 

7 

8 

6 


9 

2 

2 






Group D 

11.4 

50.0 

11.7 

16.2 

13.7 

16.5 

11.4 

12.5 

9.4 

11.2 

13.6 

12.9 

10.0 



n 

7 

2 

12 

13 

41 

92 

44 

112 

50 

85 

25 

14 

4 



Group E 

12.0 

9.2 

15.2 

13.7 

12.9 

14.8 

11.9 

12.5 

14.5 

16.0 

10.0 

2.5 

10.0 



XL 

10 

12 

31 

38 

43 

61 

59 

61 

68 

16 

13 

4 

2 



Group F 

18.3 

16.8 


13.0 

14.4 

13.3 

16.3 

13.2 

11.9 

14.4 

13.8 

10.0 

15.0 



n 

12 

31 


37 

45 

61 

60 

106 

21 

16 

13 

4 

2 


a 

Group G 

17.5 


15.6 


35.5 


14.6 


17.7 


11.1 


5.0 


& 

XL 

71 


115 


163 


118 


44 


9 


2 


§ 

Group H 

10.4 


16.5 


12.7 


12.7 


13.7 

10.5 


16.0 


0 

s 

n 

26 


68 


118 


165 


110 

43 


10 


2 

g 

Group I 

15.7 

15.2 

16.6 

15.5 

15.3 

11.9 

11,5 

15.5 

11.1 

13.0 

16.0 


10.0 


e 

n 

70 

127 

134 

139 

102 

93 

53 

42 

18 

10 

5 


1 


g 

Group J 

11.6 

12.5 

12.5 

15.4 

11.5 

13.9 

15.8 

11.4 

13.1 

12.4 

21.7 

11.0 

20.0 


1 

XL 

38 

64 

68 

124 

130 

136 

95 

90 

52 

41 

18 

10 

5 


■5 

Group K 

19.2 

20.7 

17.7 

17.0 

21.0 

16.6 

13.0 

13.5 

13.0 

16.0 

12.5 

15.0 




n 

13 

15 

39 

27 

20 

32 

43 

23 

10 

15 

4 

2 




Group L 

17.9 

17.3 

16.1 

22.3 

14.0 

14.5 

13.8 

20.0 

15.7 

20.0 

5.0 





n 

19 

22 

23 

22 

35 

31 

32 

5 

7 

1 

2 





Group M 

15.6 

14.4 

16.1 

14.8 

16.3 

12.0 

18.8 

6.7 

1L3 

0 


10.0 




u 

55 

107 

38 

44 

41 

10 

8 

3 

8 

1 


1 




Group N 

14.8 

13.9 

16.4 

16.1 

21.8 

8.9 

20.8 

15.0 

10.0 

20,0 






n 

48 

33 

42 

23 

11 

9 

12 

2 

1 

1 






Group Q 

16.7 

15.4 

14.9 

17.3 

17,6 

10.6 

14.4 

15.2 

11.5 

15.6 

10.8 

11.4 

0 



XL 

73 

90 

89 

109 

107 

87 

103 

48 

39 

25 

12 

7 

1 



Group P 

14.3 

13,4 

14.9 

14.0 

15.8 

14.6 

13.5 

14.4 

12.6 

12.5 

13.2 

12.8 

7.1 

0 


n 

53 

70 

73 

91 

90 

108 

108 

85 

102 

48 

41 

36 

7 

1 

Weiglited Aver- 
age Percents of 
the Variabllitj 















at 70 to 89 103 

302 

132 

103 

103 

95 

99 

91 

95 

91 

92 

85 

72 

0 

Sam of Weights 86 

84 

92 

89 

106 

90 

100 

77 

75 

56 

34 

26 

11 

1 
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Our results for all tests are summarized in Table 145. 
Each pair of lines of entries represents the results from 
one or more such tables as Table 142 or 143 giving the 
relative variabilities with increasing amounts of the ability 
and the weights appertaining thereto. In Table 145 all 
the entries in one column are for the same amount of abil- 
ity, to a rough approximation. That is, we arrange the 
results for any test so that it will fit the general scale from 
about a 10 percentile fourth-grade intellect to about a 95 
percentile college intellect. We have done this only ap- 
proximately because absolute precision is unattainable 
with present knowledge and because only a rough approxi- 
mation is needed for our purpose to discover any general 
relation between the amount of ability and the variability 
attached thereto, after allowance for swelling or shrinking 
of the real values of the units of the scores. 

Although all the variabilities in any column pertain to 
approximately the same amount of ability, they cannot 
properly be added by columns to show the general drift of 
Table 145. For the numbers in one row may be percen- 
tages on a base very different from that used in some other 
row. For example, the Army Exam, a numbers are on 
approximately median 9th Grade ability as a base; the 
Haggerty-Myers-Trabue combination numbers are on ap- 
proximately median 6A Grade ability as a base; the last 
seven rows arc on approximately median Columbia College 
Freshmen ability as a base. 

We shall later mass the results of Table 145 in a form 
more suitable to reveal the general relation. Table 145 
is to show the relation in each of the different tests or test- 
amalgamations. It shows no evidence of any tendency for 
variability to increase 'with ability. On the contrary, if 
one had to choose between a law of increase with amount 
of ability and a law of decrease with amount of ability, he 
would have to choose the latter. Table 145 also shows, in 
all but three of its lines, a tendency for the numbers to 
increase from the lowest extreme for a certain distance. 
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or to decrease toward the highest extreme or to do both. 
The increase comes in the low range for a test, regardless 
of whether the ability there is that of a low 4th grade 
pupil, or a low ninth grade pupil. The decrease also comes 
in the high range for a test ; regardless of what real ability 
that range covers. This is what would follow from the 
presumably common tendency of a person or committee 
devising tests, to provide rather fully tasks along the range 
of abilities which the test was designed to measure, and 
to have fewer tasks over an equally long range below or 
above that. If each task is given an equal credit in the 
score, the same real variability will of course be repre- 
sented by a smaller number in the low or in the high than 
in the middle range, supposing that abundant time is given 
to exhaust the subjects^ abilities. Table 146 gives an illus- 
tration of such a case. When the time is limited so that 
only some, or even none of the individuals do all that they 
could do, the same effect may be found. 

We have studied the facts of Table 145 in connection 
with the tests themselves in considerable detail and from 
several points of view. On the whole it seems to us that 
the most satisfactory explanation is to regard the relation 
as y = k, that is, to consider the variability of an individual 
as independent of his ability, accounting for the rises and 
falls in the curves of Table 145 by irregularities in the 
amount of real ability corresponding to one unit of score 
in a given test at different ranges of the score. We shall 
not rehearse the evidence which has impelled us to that con- 
clusion. Some of it is personal. The reader who doubts 
it should try any alternative relation, using the same cri- 
teria of merit for the two. 

The general drift of the facts of Table 145 can be stud- 
ied more easily if they are more coarsely grouped and if 
the percentages, where possible, are computed on the base 
of the variability attached to an ability of approximately 
the median ninth-grade pupiL Table 147 gives such a pre- 
sentation. In the case of the Modified Thorndike II, the 



TABLE 145. 

Th£ Belation of the Variabilitt of an Individual to His Amount of Ability; in Fifteen Tests or Amamamations or Tests. The upper 

NUMBER IS THE MEASURE OP VARIABILITY; THE LOWER NUMBER (IN ITALICS) IS THE WEIGHT ATTACHED TO IT 
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the MEASITBEMENT of INTELUEOEHCE 


Thorndike II, new style, and the Thorndike I, new style, 
with 45 minutes time, the data at or near median ninth 
j^rade ability are very scant, so we have combined the re- 
sults for these three tests giving equal weight to each and 
report them as an addition to Table 147. 

TABLE 146. The Effect of Selection of Tasks 


Task. 

Heal 
Diffi- 
culty 
of Task 

Individual 's 
Beal Average 
Ability. 

Individual ’s 
Beal 
Average 
Deviation 
from \is 
Average. 

Credit 
Given 
for the 
Task in 
the test 
Score. 

Individual’s 
Average 
Test Score. 

Individual ’s 
Average Devia- 
tion in Terms 
of the test 
Scores. 

a 

101 



1 



b 

106 


5 

1 

2 

1 

c 

111 

Ill 

5 

1 

3 

I 

d 

116 

116 

5 

1 

4 

1 

e 

121 

121 

5 

1 

5 

1 

f 

126 

126 

5 

1 

6 

n- 

g 

130 



1 



h 

131 



1 



i 

132 



1 



j 

133 



1 



k 

134 

134 

5 

1 

11 

5- 

1 

135 

135 

5 

1 

12 

5 

m 

136 

136 

5 

1 

13 

5 

n 

137 

137 

5 

1 

14 

6- 

0 

138 



1 



P 

139 



1 



q 

140 



1 



r 

141 

141 

5 

1 

18 


8 

146 

146 

5 

1 

19 

1 

t 

151 

151 

5 

1 


1 

u 

156 

156 

5 

1 

21 

1 

V 

161 

161 

5 

1 

22 

1 

w 

166 

166 

5 

1 

23 

1 


171 



1 




There is, in this coarser table, as in Table 145, no evi- 
dence that we can detect that the variability either in- 
creases or decreases in general with the magnitude.® It 

» Oonceming the effect of the error due to inaccuracy of averages men- 
tioned on page 500 we now find that no allowance need be made. The error 
might have reduced some real tendencji but there is so little evidence of any 
tendency that we cannot tell whether the reduction has been from a tendency 
for the variability to increase, or from a tendency for it to decrease. Not 
knowing in which direction to make the correction, we need not make any. 
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could be argued that it increases a trifle over low levels 
of intellect and decreases a trifle over high levels of intel- 
lect, but the facts presented in Chapter VII make it much 
more probable that the tasks of these tests were too scant, 
or the credits too low, or both, at the easy and hard ex- 
tremes of the tests- 

We thus have grounds for replacing cumbrous and am- 
biguous measures of unreliability of a test in the shape of 
its self correlations, by the simple probable error of the 
measurement. An adequate determination of this will 
suffice anywhere along the range from a dull 10-year-old to 
a gifted 20-year-old. We also have validated a new method 
of measuring the real differences signified by differences 
in arbitrary scores. That transmutation of arbitrary 
scores is best (other things being equal) which makes the 
variability of an individual the same for all levels from 
that of a dull 10-year-old to that of a gifted 20-year-old. 
We have further made a step toward using the arrays of 
a distribution separately in measuring either the real dif- 
ferences of total scores or the real differences of single 
tasks in difficulty. The form of distribution of an array 
is determined by three causes, the form of distribution of 
the average abilities of individuals in the group in ques- 
tion, the form of distribution of the variations of an in- 
dividual around his own average, and the relation of the 
variability of an individual to his average ability. We 
have previously shown that the variability of an individual 
is symmetrical and roughly of the form of the normal 
probability surface. If it is also of equal magnitude irre- 
spective of the amount of the ability, scaling tasks by what 
we may call the array method has many advantages over 
the methods hitherto used. 



APPENDIX III 


On the Form of the' Distribution of Inteixecjt in the 
Sixth Grade, the Tweufth Grade, and 
Among Coulbge Freshmen 

We have shown elsewhere^ that the form of distribution 
of intellect in the ninth grade conforms closely to the so- 
called normal curve of error. This is the report of a series 
of similar investigations into the manner in which intellect 
varies in the sixth grade, the twelfth grade, and among col- 
lege freshmen. 

It has been commonly assumed, whenever tests or tasks 
have been sealed for difficulty in a group of given educa- 
tional status, that the form of the distribution of intellect 
within such a group was truly represented by the normal 

1 

or Gaussian curve, whose equation is y— 

It is of practical importance to test the validity of this 
assumption if the scaling of tasks for difficulty is to be upon 
a sound basis. 

THE form of distribution OF INTELLECT IN THE SIXTH GRADE 

Our data for this grade consist of a series of frequency 
tables of the scores made by sixth-grade pupils in many 
different cities of the United States on eleven well-known 
intelligence examinations. They are listed below with the 
size of the population for each examination. The sources 
of the data appear on page 522 . 

From these data we have plotted the percentage dis- 
tributions for each test, from the mean as central tendency, 

1 E. li, Thorndike and E. O, Bregman, J ournal of Educational Research, 
November, 1924, the form of Distribution of Intellect in the Ninth 

Grade, 
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Test n 

1. Otis Advanced Examination 5952' 

2. National Intelligence Examination, A 1668* 

3. Trabue Completion Exercise A 1454* 

4. Pintner Non-Language Examination 1377^ 

5. Pressey Mental Survey (Cross-out) E 1057“ 

6. Hagggerty Delta 2 916* 

7. Army Examination A 742^ 

8- Myers Mental Measure 724* 

9. Illinois Examination 588® 

10, I. E. R. Tests of Selective and Relational Th inkin g, 

Generalization and Organization 379'® 

11. Army Alpha 281" 


1. Otis, A. S., Manual of Directions. 

Coxe, W. W., Variation in General Intelligence, Jour. Ed. Re- 
search, 1921, vol. 4, pp, 188-189. 

2. Data of the N. I. T. Committee supplied by Professor Guy M. 

Whipple. 

3. Trabue, M. R., Completion Test Language Seales, p. 8, 

4. Unpublished data supplied by Professor Rudolph Pintner. 

5. Pressey, S. L., A Brief Group Scale of Intelligence for Use in 

School Surveys, Jour. Ed. Psy., vol. 11, p. 96, 1920. 

6. Madsen, J. N., Intelligence as a Factor in School Progress, 

School and Society, 1922, vol. 15, p. 285. 

7. Memoirs — The National Academy of Sciences, vol. 15, Table 

187, p. 537 and Table 20, p. 334. 

8. Layton, L. H., Myers, G, C., and Myers, C. E., Group Testing 

in Altoona, Pa., School and Society, vol. 13, p. 624. 

9. Data supplied by Professor W. S. Monroe, Univ. of Illinois. 

10. Unpublished data of the Institute of Educational Research, 

Division of Psychology, Teachers College. 

11. Data of the Kansas State Teachers College, Emporia, Kansas. 

and in units of one-tenth of the standard deviation of each 
distribution. The plots appear in Figures 76 to 86. 

These individual curves are more or less irregular in 
outline and of no very constant form. On the whole the 
curves for the larger populations are the smoothest and 
most regular. 

We are not concerned, however, with the form of dis- 
tribution based upon any single test or examination. The 
form of any such single distribution, granting that the 
sample was both representative and numerically adequate, 
might not reflect the true form of distribution of intellect 




M>-» -JO -ZO -lO O 

Fig. 77. Percentage distribution of sixth-grade scores in the National Intel- 
ligence Examination, Form A. n = 1668. 



tion Exercise A. ii=:1454. 




Survey (Cross Out) E. n = 1057. 



Delta 2 Examination, n = 916. 



Fig. 84. Percentage distributioxL of Bixth-grade aeorea in the lUinoia Examina* 

tion. n = 588. 
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in this grade, either as the resTilt of the error of measure- 
ment in the individual scores, or through the effect of in- 
equalities in the units of the tests. 

In so far as inequalities in the units of the tests occur 
purely by chance, however, inequalities in one direction in 
one test will tend to be balanced by like inequalities in the 
opposite direction in some other test. 



Organization. ii = 379. 



ns 281. 
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We have therefore combined the eleven separate distri- 
butions, equal weight being attached to, each, into a single 
composite distribution, by averaging the frequencies for 
each successive one-tenth sigma, and plotting the resulting 
curve. 

It is shown in Figure 87. For purposes of comparison, 
the theoretical Gaussian curve has been dotted in. 



Fio. 87. Composite curve for the sixth grade^ based upon eleven single curves. 

The broken line indicates the theoretical normal curve. 

The two conform closely. The fit of the observed curve 
to the theoretical has been numerically determined by Pear- 
son’s Goodness of Fit Method,® — P = .999999. 

In this grade, however, it is possible that inequalities in 
the units of the tests, although present, do not occur by 
chance. The sixth grade approximates the middle region 
of the range of ability for which these tests have been de- 
vised. The normal curve bears an excellent reputation 
in psychological literature. One might conjecture with 
some show of reason, therefore, that in the construction of 
these tests there has been a more or less general and con- 
scious effort to adjust the units of the tests so as to dis- 
tribute pupils according to the normal curve, and that since 
the sixth grade approximates the mean of the range of 
ability for which the tests have been generally devised, such 

2 For the method see Tables for Statisticians and Biometxieians, edited by 
Karl Pearson^ Intro, pp. XXXI— XXXIII and Table XII. 
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deliberate inequalities would probably be most effective in 
and near the sixth grade. 

Figure 87 would not then bear evidence as to the in- 
trinsic nature of the distribution of intellect in grade six, 
but possibly in the psychologists who are responsible for 
the tests only to the skill in producing a given form of the 
tests. We should expect such skill to have its limits, how- 
ever. It seems highly improbable that it would extend to 
having the same inequalities produce a spurious symmetry 
not only in the neighborhood of the sixth grade, but through- 
out the whole range of ability for which the tests are in- 
tended. In Grade 9, for instance, it seems reasonable to 
expect that the potency of any such hypothetical inequali- 
ties to produce symmetry would have largely, if not com- 
pletely, vanished. If we find, then, that the same tests that 
display a normal curve in Grade 6 display the same form 
of distribution in Grade 9, we may justly conclude that 
there are no concerted inequalities in the units of the tests 
to which the symmetry observed may be attributed. 

We have therefore plotted ninth-grade distributions® 

3 The ninth -grade population and source for each test are as follows: 

Test n Source 

Otis Advanced Examination 3627 Otis, A.S., Manual of Directions, '21 
l^ational Intel Ex. A 494 Unpublished data of the X.E.E. 

Trabue Completion Ex. A. 273 Trabue, M. R., Completion test Language 

Scales, p. 10 

Pintner Non-Language Ex. 258 Unpublished data supplied by Prof. Ru- 
dolph Pintner 

Pressey Mental ‘ Survey Pressey, S. L., Jour, Ed. Pay., vol. XI, p. 96 

(Cross Out) E 303 

Haggerty Delta 2 2648 Compiled from unpublished data of the 

I.E.R. and from records supplied by Dr. 
W. J. OsbuTU, Wisconsin Dept, of Ed. 
Army Examination a 805 Memoirs National Academy of Sciences, 

vol. 15, Table 25, p. 344, Table, 187, p. 
537 

Myers Mental Measure 311 Myers, C. E. and G. O., Measuring Minds, 

p. 24 

Illinois Examination 380 Unpublished data of the I.E.B. 

I.E.R. Tests of Selective and Unpublished data of the I.E.R. 

Relational Thinking, Gen- 
eralization and Organization 3214 

Army Alpha 1721 Cobb, M. V., J. Ed. Psy., Nov., '22, Table 

IV 
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for the same series of tests which constitute the sixth>grade 
data, and have combined these into a composite curve in 
the same manner as already described. This curve is 
shown in Figure 88. It is as close a fit to the theoretical 
curve as Figure 87, the sixth-grade curve. P for Figure 
88 is unity. 



Fig. 88. Composite curve for the ninth grade, based upon single curves for 
the same eleven tests from which the sixth grade composite (Fig. 87) was 
derived. The broken line indicates the theoretical normal curve. 


The symmetry and goodness of fit observed in the dis- 
tribution of intellect in the sixth grade cannot therefore 
be due to consistent inequalities in the units of the indi- 
vidual tests. 

One factor remains other than the real nature of ability 
in Grade 6, to which the symmetrical form of Figure 87 
may still be due. This is the effect of the error of mea- 
surement in the individual scores. The effect of this may 
be to produce a spurious appearance of symmetry and 
normality.*' For example, if all the pupils in this grade 
were of absolutely equal ability, measurement of them with 
fallible instruments would result in a distribution of scores 
resembling the normal curve of error ; and if their teal dif- 
ferences were such as to produce a right triangle as the 
distribution, the error might largely mask this. If these 
children actually vary in ability according to the normal 
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curve of error, diminishing the errors of measurement 
should not lessen the symmetry of the group. As a series 
of measurements of a group grew less fallible, we should 
expect that the original symmetry, if spurious, would be 
accordingly diminished, but if intrinsic, would be main- 
tained. 

We have attempted to discover the effect of reducing 
the error of measurement upon the form of distribution 
of 216 sixth-grade children. Each child had been tested 
with six different intelligence examinations as follows:^ 

Test 

1. Haggerty Delta 2 

2. Kelley-Trabue Completion 

3. Myers Mental Measure 

4. Otis Advanced Examination 

5. National Examination B 

6. National Examination A 

The sum of an individuals scores in two of these tests 
should give a better measure of that individual’s ability 
than either test alone, and successive summation of the re- 
maining test scores should result in progressive improve- 
ment in the measures. The error of measurement pre- 
sumably grows less and less. If such symmetry as plots 
from the distribution of scores on the single tests show is 
spurious and exists solely by virtue of the error of mea- 
surement in those single scores, then plots from the sum- 
mated scores, in which the error is progressively dimin- 
ished, should grow less and less symmetrical. 

Such is not the case. Figures 89 to 94 are plots of the 
216 scores in each of the six tests. The fit of each to the 
theoretical normal distribution is given in Table 14B. 

Figures 95 to 99 are plots of the distributions obtained 
by successive addition of the test scores in the order in 
which the tests arc listed in Table 148. 

^ We are indebted to Dr. J. L. Stenquist for these unpublished data. 
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TABIiE 148. 

The Oloseitbsb or Fit or Six Test Scoses, Taken Sikoly. 


Test 


p 

Haggerty Delta 2 


J570 

Myers Mental Measure 


.272 

Kelley Trabue Completion __ 


.738 

Otis Advanced Examination 


.496 

National B 


.543 

National A 


.067 







Figs. 89 to 94. Distributions of 216 sixtb-grade pupils' scores iu various 

examinations. 

Fio. 89. Haggertj Delta 2. 

Fig. 90. Myers Mental Measure. 

Fig. 91. Eelley-Trabue Completion. 

Fig. 92. Otis Advanced Examination. 

Fig. 93. National Int. Ex. B. 

Fig. 94. National Int. Ex. A. 
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The fit of each of these to the theoretical is given in 
Table 149. 

There is in these graphs and in these figures no such 
consistent change, paralleling the change in the error of 
measurement, as would justify us in attributing such sym- 
metry as the group shows to the error of measurement. 



Fiqs. 95-99* Distributions of 216 sixth-grade pupils’ combined scores. 
Fig. 95. Haggerty Delta 2 and Myers Mental Measure combined. 

Fig. 96. Haggerty Delta 2, Myers Mental Measure and Kelley-Trabue Com- 
pletion, combined. 

Pig. 97. Haggerty Delta 2, Myers Mental Measure, Kelley-Trabuc Completion 
and Otis Advanced Examination, combined. 

Pig. 98. Haggerty Delta 2, Myers Mental Measure, Helley-Trabue Completion, 
*Otis Advanced Examination, and National Intelligence 
Examination B, combined. 

Pig. 99. Haggerty Delta 2, Myers Mental Measure, Kelley-Trabue Comple- 
tion, Otis Advanced Examination, National Intelligence Examination 
B, and National Intelligence Examination A, combined. 
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When Haggerty and Myers scores are combined the fit 
is .376 as compared with an average of .421 for them taken 
separately. When Haggerty, Myers and Kelley-Trabne 
are combined the fit is .706 as compared with an average of 
.526 for them taken separately. When Haggerty, Myers, 
KeUey-Trabue and Otis are combined the fit is .547 as com- 
pared with an average of .519 for them taken separately. 
When Haggerty, Myers, Kelley-Trabne, Otis and National 
B are combined the fit is ,748 as compared with an average 
of .524 for them taken separately. When all six are com- 
bined the fit is .353 as compared with an average of .471 for 
them taken separately. 

TABLE 149, 

The CiiOSENESS ot Fit of Six Test Scoees Taken Two or More at a 

Time and Arranoed. 


Tests P 


Haggerty and Myers .376 

Haggerty^ Myers and Kelley-Trabne ~ .706 

Haggerty, Myers, Kclley-Trabue and Otis - .547 

Haggerty, Myers, Kelley-Trabne, Otis and National B .748 
Haggerty, Myers, Kelley-Trabne, Otis, National B and 

National A .853 


We may conclude therefore that intellect in Grade 6, if 
measured in truly equal units, varies in general accordance 
with the normal probability curve. Intellectual tasks may 
therefore be scaled for difficulty in this grade by this 
hypothesis, with close approximation to the truth. 

TBB rOBM OF TJHE DISTBIBUTION OF INTELLECT IN THE 

TWELFTH GBADE 

For the twelfth grade we have a series of ten frequency 
distributions representing the scores made by twelfth- 
grade pupils in as many different intelligence examina- 
tions. 

The examinations are listed below, with the number of 
pupils who took each examination and the sources from 
which the data were obtained. 
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Examination n 

1. Terman Group Test of 4886 

Mental Ability 

2. Brown University Psycho- 3333 

logical Examination 


3. I.E,R. Tests of Selective 26385 

and Belational Think- 
ing, Generalization and 
Organization, Porms A 
and B 

4. Thorndike Intelligence 1627 

Examination for High 
School Graduates, Part 
1, Forms T> and N 

5. Army Alpha Examination 1387 


6. Otis Group Intelligence 1226 

Scale Advanced Exami- 
nation 

7. Strickland Test 1020 


8. Mentimeter Scale 874 


9. Miller Mental Ability 739 
Test 


10. Haggerty Intelligence Ex- 668 
amination, Delta 2 


Source 

Manual of Directions, page 9. The 
World Book Co. (The table of per- 
centile scores was converted into a 
frequency table) 

The Intelligence of Seniors in the 
High Schools of Massachusetts. 
Stephen S. Colvin and Andrew H. 
MacPhail. Bulletin, 1924, No. 9. 
Department of the Interior, Bureau 
of Education. Page 14 
Unpublished data of the Institute of 
Educational Besearch 


A Mental Educational Survey. G. M. 
Bueh. University of Iowa Studies 
in Education, Volume 2, No. 5, 
page 22 

The Limits Set to Educational 
Achievement by Limited Intelli- 
gence. Table VII. M. V. Cobb, 
Journal of Educational Psychology, 
November, 1922 

Manual of Directions, page 60. 
World Book Co. 

This is a two-hour examination. The 
frequency distribution was supplied 
by the kindness of Professor V. L. 
Strickland, Kansas State Agricul- 
tural College, Manhattan, Kansas 
Monthly Bulletin, Bureau of Educa- 
tional Besearch, University of North 
Carolina, July 21, 1923 
Manual of Directions, page 17. 
World Book Co- The percentile 
distribution was converted into a 
frequency distribution 
From data supplied through the 
courtesy of Dr. W. J. Osbum, 
State Department of Education, 
Wisconsin 


5 1666 for Form A, 972 for Form B. Plots were made for each form sepa- 
rately and were then combined into a single distribution by averaging. 
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The frequency distributions of these tests have been 
converted into percentage distributions, and combined into 
one composite distribution in the manner already described 
in Section 1. The individual percentage distributions are 
shown in Figures 100 to 109. 



Pio. 100. Percentage distribution of twelfth grade scores in the Terman Group 

Test of Mental Ability. n = 4886. 



Fia. 101. Percentage distribution of twelfth-grad© scores in the Brown Uni- 
versity Psychological Examination. n=:333. 

The composite twelfth-grade distribution appears in 
Fig. 110. P for this equals .999911. 

Since it is hardly likely, in view of the evidence which 
has already been presented in connection with the sixth 
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grade and the ninth grade and which will be presented in 
the next section for college freshmen, that the normality 
and symmetry of Figure 110 result from the error of mea- 
surement, we may conclude that intellect in the twelfth 
grade is also distributed in conformity to the Gaussian 
curve. 



Tests of Selective and Belational TMnking, Generalization and Organization. 
Total n = 2638. (This curve is an average of the separate curves for Forms 
A and B.) 



Examination for High School Graduates, Part I, Forms 
D and N. N = 1527. 




Zte. 106. Percenter diotribntion. ot twdfth-grade scores in tlie*Striefclit&d 


Test, ns: 1020. 
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Fig. 108. Percentage distribution of twelfth-grade scores in the Miller Mental 

Ability Test, n = 739. 

THE FORM OF THE DISTRIBUTION OF INTEULECT AMONG 
COULEGB FRESHMEN 

The composite distribution for college freshmen is 
based upon percentage distributions plotted from fre- 
quency tables for the following examinations:® 

® Whenever two or more groups, whether for the same or different forms 
of the examination, are listed under any examination, separate percentage dis- 
tributions have been computed for each group so listed. These have then 
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Examination 
!• Armj Alpha 


n Source 

2545 Universitj Students Intelligence Rat* 
ings According to the Army Alpha 
Test. £. L. Noble and George F* 
Arpa« School and Society, Yolume 
11, page 234. 

400 Intelligence Tests of Yale Freshmem 
J. £. Anderson. School and So- 
ciety, Volume 11, page 419 



Intelligence Examination, Delta 2. n = 668. The composite twelfth-grade 
distiibution appears in Figure 110. P for this curve equals .999911. 



been combined by averaging, equal weight being attached to each group, into 
a single distribution for each examination. Thus no examination receives 
greater weight than any other in the final composite distribution. 
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Ezamiimtion xi 

2. Brown UniTeraity Psyeho- 2118 
logical Examination 


3. Arxnj Examination A 701 


4. Iowa Comprehension Tests 

B-1 1046 

D-1 1085 

5. Minnesota Becognition Yo- 1208 

cabulary. A— 2 


6. Morgan Mental Test 1250 

7. Princeton Examination, €23 

Series II 

8. Smith College Entrance 

Examination No« 1 371 

Examination No. 2 486 

Examination No. 3 604 

Examination No. 4 596 

8. Thorndike Intelligence 
Examination for High 
School Graduates 
Part I, Form B 1085 

Part I, Forms J and K 1046 

Part I, Forma E and F 834 
and Fart II, Form C 
Total score Smith 525 

College freshmen 
Total score, Columbia 356 

freshmen 

Part VI, Form C 272 


Thorndike Intelligence 
Examination for High 
School Graduates 
Part I, Forms E and J, 

Part II, Form C and 241 
Part III, Form AA 
10. Thurstone Psychological 5495 
Examination, Test IV 
Liberal Arts freshmen 


Source 

Distribution of scores made by six 
freshmen classes at Brown Uniyer* 
sity. Data supplied by the courtesy 
of Dr. A. H. MacPhail 
Memoirs of the National Academy of 
Sciences, Volume 15, page 537, 
Table 187. 

Data supplied by the courtesy of Dr, 
G. M. Buch 

Data supplied by the courtesy of Dr. 
M. E. Haggerty and Dr. D. G. 
Paterson 

Data supplied by the courtesy of Dr. 
J. J. B. Morgan 

Data supplied by the courtesy of Dr. 
C. C. Brigham 

Distribution of scores made by four 
freshmen classes at Smith College. 
Data supplied by the courtesy of 
Dr. D. C. Bogers 

Data supplied by the courtesy of Dr. 
G. M. Buch 


Data supplied by the courtesy of Dr. 

Agnes L. Bogers 
Unpublished data of the I.E.B. 

Data supplied by the courtesy of Dr. 
B. M. Smith 


Unpublished data of the I.E.B. 

A Cycle Omnibus Test for College 
Freshmen. L. L. Thurstone. Jour* 
nal of Educational Besearch, 1921, 
Volume 4, Table 2 
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Examination 

n 

Sonrce 

Yale Examination 


Data supplied by the eourtesy of Dr. 

Preliminary Form 

647 

J. E. Anderson 

Forms 1 and 2 

815 


Forms 1 and 2 and 3a 
or 3b 

829 


Forms 1, 2 and 4 

820 



The plots for each examination are shown in Figures 
111 to 121. 

The composite obtained by averaging Figures 111 to 
121 is presented in Figure 122. Its fit to the normal curve 
is expressed by P = .999988. 



Fig. 111. Percentage distribution of eollege-freslimen scores in Army Alpha. 
Composite of separate curves for two groups. Total n=:2945. 
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nation A. n =: 701. 




Fig. 114. Percentage distribution of college-freshmcn scores in Iowa Com- 
prehension Tests. Composite of separate eurves for Forms 
B-1 and D-1. Total n = 2131. 



Beccgnition Yoeabulaxy^ A-2. n=:1208. 


Wfil 




nro 




Fig. 116. Percentage distribution of college-freshmen scores in Morgan Men- 
tal Test. n = 1250. 




30 T5 io ZTJ w 

Flo. 119. Percentage distribution of college-freshmen scores in Thorn- 
dike E xam i n ation for High School araduates. Composite of separate curves 
for seven groups, many different forms. (See tabulation, page 540. Total 
n = 4359. 
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THE EFFECT OF THE EBROB OF MEASUBEMEHT 

We have, for each of six groups of college freshmen, 
several series of test scores. 

For four of these groups, which we have called Groups 
1, 2, 3 and 4, we have a series of five scores for each student. 
Three of the five scores represent each student’s record on 
three major parts of one long examination, or on two parts 
of one examination, and the total score on a second ex- 





Fiq. 122. Composite curve for college-freshmen, derived from eleven single 
examination curves. The broken line indicates the 
theoretical normal curve. 


amination. The fourth and fifth scores represent succes- 
sive summation of the first two, and all three single scores, 
respectively. 

For the remaining two groups, which we have called 
Groups 5 and 6, we have, for each student, three sets of 
scores. The first two represent performance on two major 
parts of a single examination, or on two separate examina- 
tions. The third score is the sum of these two single 
scores. 

The examinations performed by each group, the size 
of each group and the sources of the data are tabled below. 

These data enable us to observe the effect, upon the 
form of the distribution, of progressively diminishing the 
error of measurement. If the symmetry and normality 
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Examination 

A Thorndike Int. Exam, for 
Part I, Form J 

B Thorndike Int. Exam, for H.S.G.j 
Part I, Form K 

O Iowa Comprehension Exam. B-1 

2. A Thorndike Int. Exam, for H.S.G., 

Part I, Form E 

B Thorndike Int, Exam, for H.S.G., 
Part I, Form F 

C Thorndike Int. Exam, for n.S.G., 
Part II, Form C 

3. A Smith College Ent. Exam. Form 

3, Part 1 

B Smith College Ent. Exam. Form 
3, Part 2 

C Smith College Ent. Exam. Form 
3, Part 3 

4. A Princeton Examination Series V 
B Princeton Examination Series VI 
O Princeton Examination Series VII 


6. A Minnesota Recognition Vocabulary 
A~2 

B Minnesota Opposites and Comple- 
tion of Definitions 


Series Source 

1046 

Data supplied by the cour- 
tesy of Dr. G. M. Bach 


834 

Data supplied by the cour- 
tesy of Dr. G. M. Ruch 


633 

Data supplied by the cour- 
tesy of Dr. D. C. Rogers 


629 Data supplied by the cour- 
tesy of Dr. C. C. Brigham 

Data supplied by the cour- 
tesy of Dr. G. M. Ruch 

1208 Data supplied by the cour- 
tesy of Dr. M. B. Hag- 
1203 gerty and Dr. D. G. Pat- 
erson 


5. A Thorndike Int. Exam, for H.S.G., 1085 

Part I, Form B 

B Iowa Comprehension Exam. D— 1 


decrease with each successive combination of scores, they 
are in so far due to the chance error. If they do not de- 
crease with more and more combination, the symmetry and 
normality are in so far really characteristic of the group 
itself. 

We have examined the effect of squeezing out the error 
in individual measurements, by successive combination of 
test scores, upon the form of the distribution of each one 
of the six groups, and also upon the composite curves ob- 
tained by combining the similar series of distribution? for 
each group. 
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Plots of the series of single and snnmiated scores for 
each of the six groups are given in Figures 123 to 128. 

In each figure the observed frequency distribution is 
given in solid outline. The broken line gives the frequen- 
cies to be expected by the ^eory of the normal curve wher- 
ever they differ from the frequencies of fact. 






Pia. 123A. Group 1: Distribution of 1046 Freshmen scores in Thorndike 
Examination for Iligh-School Graduates, Part I, Form J* 

Fxa. 123B. Group 1: Distribution of 1046 Freshmen scores in Thorndike 
Examination for High'School Graduates, Part I, Form K. 

Fio. 123C. Gioup 1: Distribution of 1046 Freshmen scores in Iowa Compre- 
hension Examination B-1. 

Fio. 123D. Group 1 ; Distribution of 1046 Freshmen scores in Thorndike 
Examination for High-School Graduates, Part I, Form 
J plus Part I, Form K. 

Fig. 123E. Group 1: Distribution of 1046 Freshmen scores in Thorndike 
Examination for High-School Graduates, Part I, Form J plus Part I, 
Form K plus Iowa Comprehension Examination B-1. 


The fit of each observed distribution of scores, single 
and sununated, to each series of theoretical normal fre- 
quencies has been numerically determined by the Pearson 
Goodness of Pit Method. The P’s are shown in Table 150. 
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Examination of the figures and the table reveals no con- 
sistent and progressive tendency in the individual groups 
to lessened symmetry and normality with progressive re- 
duction of the error. 

In the composite curve the fit to the normal becomes, 
practically, progressively better. 



Fig. 124A. Group 2: Distribution of 834 Freshmen scores in Thorndike Ex- 
amination for High-School Graduates, Part 1, Form E. 

Fio. 124B. Group 2: Distribution of 834 Freshmen scores in Thorndike Ex- 
amination for High-Bchool Graduates, Part I, Form F. 

Fig. 124C. Group 2: Distribution of 834 Freshmen scores in Thorndike Ex- 
amination for High-School Graduates, Part II, Form C. 

Fig* 124D. Group 2: Distribution of 834 Freshmen scores in Thorndike Ex- 
amination for High-School Graduates, Part I, Form E 
plus Part I, Form F. 

Fig. 124E. Group 2: Distribution of 834 Freshmen scores in Thorndike Ex- 
amination for High-School Graduates, Part I, Form E plus 
Part I, Form F plus Part II, Form O. 


Our data give us two sets of composite distributions, 
one in which all six groups are represented, and which con- 
sists of two curves for the two sets of single scores, and a 
third distribution of these single scores snmmated, and a 
second set of distributions in which only the four groups 
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are represented, for which we have three sets of single 
scores and the two sets of snccessively snioniated scores. 

These distributions have been obtained in the same man- 
ner as all of our composite distributions, namely, by plot- 
ting percentage distributions for each frequency table, and 
deriving the composite, by 'averaging the percentage fre- 



Fig. 125A. Group 3: Distribution of 633 Freshmen scores in Smith Entrance 

Examination, Form 3, Part 1. 

Fig. 125B. Group 3: Distribution of 633 Freshmen scores in Smith Entrance 

Examination, Form 3, Part 2. 

Fig. 125C. Group 3: Distribution of 633 Freshmen scores in Smith Entrance 

Examination, Form 3, Part 3. 

Fig. 125D. Group 3 : Distribution of 633 Freshmen Scores in Smith Entrance 
Examination, Form 3, Part 1 plus Form 3, Part 2. 

Fig, 125E. Group 3 : Distribution of 633 Freshmen scores in Smith Entrance 
Examination, Form 3, Part 1 plus Part 2 plus Part 3. 


quencies for each successive l/lOo from the mean, equal 
weight being attached to each distribution. 

Figures 129, 130 and 131 present the composite distri- 
butions of the single and summated scores of all six groups. 
Figures 132 to 136 the composite curves of Groups JL to 4. 
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The broken lines in Figs. 129 to 136 indicate the ** normal** 
curve. 

The fit of each composite curve to the theoretical nor- 
mal curve is given in Table 151. 



Pig. 126A. 
Fig. 126B. 
Pig. 126C. 
Pig. 126D. 
Pig. 126E. 


Group 4: Distribution of 629 Freshmen scores in Princeton Ex- 
azmnation. Series V. 

Group 4: Distribution of 629 Freshmen scores in Princeton Ex 
ammation. Series VI. 

Group 4: Distribution of 629 Freshmen scores in Princeton Ex- 
amination, Serit's VII. 

Group 4: Distribution of 629 Freshmen scores in Princeton Ex- 
amination, Series V plus VI. 

Group 4: Distribution of 629 Freshmen scores in Princeton Ex- 
amination, Series V plus VI plus VII. 


The error of measurement is therefore not the potent 
factor in determining the normal symmetrical form of dis- 
tribution of intellect among college freshmen. We may 
then conclude that this form of distribution describes the 
actual variation of intellect in this group. 
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Flo. 127 A. Group 5: Distribution of 1,085 Freshmen scores in Thorndike 
Examination for High-School Graduates, Part Form B. 

Fio. 127B. Group 5: Distribution of 1,085 Freshmen scores in Iowa Com* 

prehensive Examination D— 1. 

Fzo. 127C. Group 5: Distribution of 1,085 Freshmen scores in Thorndike 
Examination for High-Behool Graduates, Part I, Form B plus 
Iowa Comprehensive Examination D— 1. 

Fio. 128A. Group 6: Distribution of 1,208 Freshmen scores in Minnesota 

Vocabulary Examination. 

Flo. 128B. Group 5; Distribution of 1,203 Freshmen scores in Minnesota 
Examinations, Opposites and Dednitions. 

Fio. 128C. Group 6: Distribution of 1,203 Freshmen scores in Minnesota 
Examinations, Vocabulary plus Opposites and Definitions. 
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TABLE 150. 

Gooqnkss of Fit or Obbieb.ved Dibtbibutionb — Gboups 1 to 6 — 

TO NobmaIi Cn&vE. 



P. for diB. of 
single scores 

P. for die. of sums 
of A and B scores 

P. for dis. of sums 
of A, B and O aeorea 


Beries 




Group 1 

A 

.863808 

.979015 

.681535 


B 

.340511 




o 

.000000 



Group 2 

A 

.705301 

.108533 

.965324 


B 

.001299 




o 

.523111 



Group 3 

A 

.019390 

.598101 

.279157 


B 

.411783 




c 

.000717 



Group 4 

A 

.028408 

.000038 

.003595 


B 

.000000 




0 

.006401 



Group 5 

A 

.444183 

.486321 



B 

.521361 



Group 6 

A 

.974138 

.261391 



B 

.269385 




TABLE 151. 

Gk>ODNEss OF Fit of Couposite Distributions to Nobmai. Curve. 



P. for composite 
dis. of single 
scores 

P. for composite 
dis. of sums of 

A and B scores 

P. 

dis. 

B 

for composite 
of sums of A, 
and C scores 


Beries 





Groups 1 to 6 

A 

.999981 

1. 




B 

.999765 




Groups 1 to 4 

A 

.986097 

1. 


.999818 


B 

.999713 




<1 

C 

.949824 






THB DISTRIBUTION OF INTSXUBCT BT OBADRS 


55.3 
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curveB. Sum of the A and B scores. 



Fxo. 136. Composite distribution of college^freshmen scores baaed on four 
curves. Sum of the A, B, and C scores. 


APPENDIX IV. 


The Homogeneity of Intellect CAVD at All Levei^ 

OF Difficulty 

Our question is how far the ability required to succeed 
with hard CAVD tasks differs from that required to suc- 
ceed with easy CAVD tasks by being a greater amount of 
the same kind of thing, and how far, on the contrary, it 
is qualitatively different. If we had composites at each 
level of difficulty, made up of, say, a thousand single tasks 
or enough to measure the ability at each level perfectly, 
and tested a random million of age twenty with them, how 
nearly would the correlations between different levels of 
difficulty approach 1.00; and how nearly would the remote- 
ness of one level from another approach zero in its in- 
fluence upon the correlations? 

We have to estimate the correlations for composites of 
1,000 or more from the correlations for composites of 40 
or less. This is done by the well-known attenuation for- 
mula of Spearman. We have to estimate the correlations 
in such an age population from the correlations in various 
groups of more restricted range. This may be done by 
the Pearson formulae for correction for range. We have 
to estimate the effect of remoteness over the whole range 
of difficulty from tasks like those of the 40-Composite A 
to tasks like those of the 40-Composite Q by the effect of 
remoteness of two or three steps, since none of our groups 
was tested over the whole range of difficulty. 

We present the facts from four groups. The first is 
98 adult imbeciles ; the second is 121 candidates for college 
entrance; the third is 246 pupils of Grade 9; the fourth is 
192 pupils of Grade 9. 

The 40-Composites A, B, C, and D were divided each 
into two 20-composites, by taking elements 1, 2, 3, 9, and 
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10 from C, A, V, and D to make AI, BI, etc., and by taking 
elements 4, 5, 6, 7, and 8 from C, A, V, and I) to make All, 
BU, etc. The correlations between one and another of 
these 20-composites AI, All, BI, BII, Cl, CH, DI, and 
DII in the case of 98 adults of mental age 30 months 
to 60 months, are as shown in Table 152. The self-corre- 
lations of AI with All, BT with BII, CT with CII, and DI 
with DII, are also shown there. The correlations between 
two composites of infinite length, at various diflierences of 
difficulty, may then by this determination be expected to 
be as shown in Table 153. 

The average of the correlations of neighboring com- 
posites is .94; that of the correlations of composites one 
step removed, is .86 (.84 by Pearson, .885 by Sheppard) ; 
that of the A to D correlations is .78 (.71 by Pearson, and 
.84 by Sheppard). 


TABLE 152. 

Self- and Inter Correlations or Four 40-CoMPOSiTEa or OAVD, Each 
Divided into Two Bandom Halves (I and II). 98 Imbeciles (P means 
Pearson CoErriciENT, Sh means Shfppard CoEmciENT) . 



All 

BT 

BII 

Ct 

CII 

DI 

DU 


P. Sh 

P Sh 

P SE 

P Sh 

P Sh 

P Sh 

P Sh 

A I 

.8(5 .87 

.78 .80 

.78 .78 

.67 ,71 

.73 .78 

.62 .76 

.52 .70 

A II 


,78 .76 

.78 .81 

.68 .71 

.70 .74 

.58 .71 

.49 .67 

B I 



.81 .74 

.74 .83 

.80 ,81 

.63 .76 

.59 .72 

B II 




.76 .78 

.80 .80 

.70 .70 

.62 .77 

0 I 





.81 .91 

.65 .67 

.64 .79 

C II 






.73 .70 

.74 .80 

D I 

D II 







.69 .83 


In these and in similar correlations between levels at 
different degrees of remoteness, it should be kept in mind 
that the range is very restricted, and that if all Americans 
of the same chronological ages as these 98 imbeciles had 
been measured by A, B, C and D, the correlations would 
have been very much higher. The a of the group of 98 
adults was about 8 mental months. That of the group of 
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‘‘all Americans of comparable chronological ages^’ would 
be at least four times as great/ 

In a group with a variability four times that of the 
group of 98 imbeciles, the correlations corresponding to 
the .94, .86, and .78 would be approximately .99^, .99, 
and .98. 

TABLE 153. 

THB iNTER-COBBKIiATIONS OF POtTE CAVD OOUPOSITS TASSS LIKB A, B, C, 
AND D IN CONSTITTmON AND DlPPICm/TY, BUT EacH CSONSISTINO OF 
AN Infinitbdt Liasoe Nttmbeb of Singlib Tasks. Thk Inteb* 
COBBKIiATIONS OF TABLE 152 COBBECTED FOB ATTENUAXION. 



B 

C 

D 



P Sh 

P Sh 

P Sh 


A 

.93 .98% 

.83 .83 

H 

bo 


B 


.96 .98 

.85 .94 


0 



.92 .85 



TABLE 154. 

Self- and Inteb-Corbelations op Pode 40-Oomposites op OAVD, Each 
Divided into Two Random Halves (I and H) 121 High School 

GRADDATES. (P MEANS PBAESON, SH MEANS SHEPPARD.) 


NI NH 

P Sh 

OI on 

P Sh P Sh 

PI 

P Sh 

PII 

P Sh 

QI 

P Sh 

Qn 

P Sh 

NI 

.57 .46 

.64 .57 .68 .62 

.55 .57 

.61 .65 

.54 .59 

.47 .37 

Nil 


.64 .62 .61 .62 

.58 .62 

.63 .65 

.48 .37 

.52 ,41 

OI 


.66 ,71 

.65 .68 

,60 .63 

.60 ,65 

.43 .19 

Oil 



.60 .68 

.70 ,75 

.55 .62 

.53 .39 

PI 




.62 .75 

.54 .48 

.50 .48 

PIT 





.46 .39 

.58 .53 

QI 






.49 .47 

QII 








Similar facts for a group of 121 candidates for entrance 
to college, measured with 20-composites formed by divid- 
ing N, O, P, and two 15-eompo sites formed by dividing 
Q, are slaown in Tables 154 and 155. Here the average 

1 The a of the random sample of 653 men in the draft from 9 camps was 
34 mental months. [Memoirs^ p. 391.] 
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correlations are 1.02^ (.995 for P and 1.05^ for Sh), 
1.00% (.96 for P and 1.05 for Sh), and 1.14 (.95 for P and 
1.34 for Sh) in order of remoteness. 

TABLE 155. 

THX lNT&K-COBBXiI.^TIONB OF FOXTB CAVD CkiMPOSlTi: TASKS LiKS N, O, P 
AND Q, IN CJONSTRtrCTION AND DlFFICUiyrr, BUT EACH (ToNSISTINa 
OF AN iNFINTTKIiT LABOE NUMBER OF SINGLE TASKS. 

121 High School Oraduates. 



o 

P 



Q 


P Sh 

P 

Sh 

P 

Sh 

N 

1.05 1.061/6 

.99^t 

1.05 

.95 

1.34 

O 


.991/6 

.94 

.92 

1.05 

P 




.94 

1.15 


The 40-composites I, J, K, L, and M, showed the inter- 
correlations of Table 156 in the case of 246 pupils in 
Grade 9. The self-correlations when we divide each 40- 
composite into two 20-composites made of 5C, 5A, 5V, and 
5D taken at random were : 



P 

Sh 

I 

.53 

.56 

J 

.63 

.66 

K 

.481/2 

.59 

L 

.75 

.75 

M 

.59 

.53 

The self-correlations of the 

respective 40-composites, 

each with another 40-composite 

of the same difficulty are 

then by Spearman’s 

formula 

= :r , ^ las shown below. 

1 + Ti/ 


P 

Sh 

I 

.70 

.73 

J 

.77 

.80 

K 

.65 

.74 

li 

.86 

.86 

M 

.68 

.69 
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The correlations between two composites of infinite 
length, at various differences of difficulty, derived from 


Table 156 by using r^a 
Table 157. 


Taibi 


'\/rAlA2 rBlB2 


are as shown in 


TABLE 156. 

The ^Raw' Intee-Cokbelations op Five CAVD 40-CIo3cfosites nr 246 

Pupils op Qbadb 9. 



J 

Sh P 

K 

Sh P 

L 

Sh 

P 

M 

Sh P 

I 

.59 

.53 

.62 


.49 

J 


.70 .66 

.72 

.66 

.72 

K 



.67 

.66 

.69 

L 





.63 .67 

M 







TABLE 157. 

The Intee Coerelationb op Five CAVD Composite Tasks, Like Those op 
Table 156 in Constitution and Dippiculty, but Bach Consist- 
ing OF AN INPINITELT LARGE NUMBER OP SINGLE TASKS. 



Sh P 

Sh P 

L« 

Sh P 

M« 

Sh P 



.77 

.72 


.78 


.69 


Jc. 


.91 

.93 

.87 

.81 

-97 


K„ 




.84 

.88 

.96% 


L« 






.82 .87 



The average of the correlations of neighboring compo- 
sites in this group is .85; the average of composites one 
step removed is .84; that of composites two steps removed 
is .871/^; the correlation between the two which are three 
steps removed is .69. 

The composites K, L, M, and N, showed the inter-corre- 
lations of Table 158 in the case of 192 pupils in Grade 9. 
The self-correlations, using 20 with 20, are in order: .67, 
.76, .57, and .58 using Sheppard’s formula, and .58, .80, 
.64, and .62, using Pearson’s formula. 
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The correlation between a 40-eomposite and another 40- 

2ri \ 

composite of the same difficulty are then (by ^2 — j'qTY’y 
as shown below; 



Sh 

P 

K 

.80 

.73 

L 

.86 

.89 

M 

.73 

.78 

N 

.73 

.77 


TABLE 158. 

The "Raw” iNTBR-CoERELATioiirs or Pour OAVD Composites in 192 Pupii.s 

OP Grade 9. 




L 

Sh P 

M 

Sh P 

N 

Sh P 



K 

.47 .61 

.59 

.69 

.59 

.61 



L 


.77 

.73 

.50 

.57 



M 




.68 

.71 



TABLE 159. 

The rNTEB-OoKREriATiONS OP Poor CAVD Composites Like Those op Table 
158 IN Constitution and Dipfioulty, but Each Consisting op an 
Inpinitelt Large Number op Single Tasks. 



L, 

Sh P 

M« 

Sh P 

N« 

Sh P 


K- 

L« 

.56 .75 

.77 .91 

.97 .88 

.73 .81 

,63 .69 

.93 .92 



The correlations for composites of infinite length, at 
the various differences of difficulty, derived from Table 

158 by using rAB= — _ are shown in Table 159. 

'\/TaiA2 TbiB2 

For this group of 192, the average of the correlations of 
neighboring composites is ; that of the composites one 
step removed is .75; that of the two which are two steps 
removed is .77. 
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Table 160 summarizes the facts for all four groups. 
We correct for restricted range in the groups of high- 
school graduates and pupils in Grade 9, by taking the varia- 
bility of age 20 to be twice that of high school graduates 
or of pupils in Grade 9. The variability of the literates of 
the draft who took Alpha was If times that of pupils in 
Grade 9 or of pupils in Grade 12 tested with Alpha. If 
the illiterates at one extreme and the officers at the other 
had been included, the variability would have been greater. 


TABLE 160. 

SUMMABT OF INTER CJOHBJELATIONS CORBECTED FOR ATTENUATION* 




Neighboring 

Composites 

Composites 
One Step 
Removed 

Composites 
Two Steps 
Removed 

Composites 
Three Steps 
Removed 

98 imbeciles 


.94 

.86 

.78 


121 H. S. graduates 


1.02% 

1.00% 

1.14 


246 9tli grade 


.85 

.84 

.87% 

.69 

192 9th grade 


.85% 

.75 

.77 


Average 


.91 

.86% 

.89 

.852 


Corrected for 

Restricted Range 


98 imbeciles 


.99% 

.99 

.98 


Average of the other 





groups 

Weighted average 

of 

•97% 

.96 

.983 


all four groups 


.98 

.96% 

.984 



The variability of 63,647 enlisted men in four camps in 
Army Examination a was almost If times that of pupils 
in Grade 9, and almost 2 times that of college freshmen. 
Here also the inclusion of the illiterates and officers would 
have raised the variability of the adult group. 

We have in these four determinations, taken together, 
a proof that the effect of remoteness in difficulty upon the 
inter-correlations of various CAVD composites is small, 
and a rough measure of how small it is. 

2 When .69 is averaged witb the figures in preceding column. 

> .96 if the .69 is used in the average. 

♦ .96% if the .09 is used. 
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The change as we pass from neighboring composites to 
composites one step removed in difficulty is — .08, — .01, 
— .08^, and — .02, with an average of — .05 which has a 
0t-<» of ± .034. The change as we pass from neighboring 
composites to composites two steps removed is — .16, 
+ .02^, — .06i, and + -lli, with an average of — .02, 
which has a Ot-o of ± .102. If we count the three-step 
case where the change is — .16 (.85 to .69) in with the two- 
step cases, we have an average of — .05 with a Ot-o of 
± .107. If the effect of remoteness were large, twice the 
amount of remoteness would have a greater relative effect, 
and the effects would all be larger relative to their unre- 
liabilities. One step of remoteness changes the correlation 
by — .08, — .01, — .08^, — .02, — .08, -j- .13J, -f- .03J, -j- .02, 
and — .18^. The average of these, — .03, has a Ot-o of 
± .086. So there is a probability of 36 in a hundred that 
remoteness raises the correlations, and a probability of 64 
in 100 that it lowers them. 

When the correlations are corrected for the restriction 
of range, the correlations are around .97 and the average 
drop for one step of remoteness is .00^. 

It is hard to state in any concise fashion how much of 
a difference in difficulty this obtained drop of .03 corre- 
sponds to. We may best simply list the changes in the 
percent of successes to which the facts which it averages 
correspond. They are : 

In the imbeciles, from 48.3 to 12.8 

“ 12.8 to 6.0 

In the college graduates, from 48.1 to 27.5 

‘‘ 27.5 to 3.7 

In the 246 of Grade 9, from 89.4 to 61.4 

61.4 to 32.9 
“ 32.9 to 5.3 

In the 192 of Grade 9, from 16.3 to 7.2 

‘‘ 7.2 to 1.1 

It is somewhat strange that since the drop is so small, 
the correlation between neighboring composites should not 
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be around .97 and all the obtained correlations nearer to 
unity than they are. This may be just a matter of chance 
due to the small populations. Or there may be factors of 
carelessness and lack of zeal on the part of some of the 
subjects, and even some undetected cheating, which the 
correction for attenuation may not properly allow for. 

This and other imperfections in the determinations do 
not, however, impair the essential result that the correla- 
tions between composites far apart in difficulty are little, 
and possibly not at all, lower than the correlations between 
neighboring composites. The series of CAVD composites 
from one so easy as the 1 to one so hard as the Q do mea- 
sure much the same sort of thing. We do have a right to 
call it by one name and to measure increases in it by the 
series of cardinal numbers. 



APPENDIX V. 


The Adequacy of Tasks of Ahy One Level of Difficulty 
AS A Measuee of All of Iisttellbct CAVD 

We wish to know how nearly certain CAVD composite 
tasks would measure all of intellect CAVD and nothing but 
intellect CAVD, if the number of elements in the tasks were 
extensive enough to make its own self-correlation 1.00. 
The answer is given by the correlation corrected for atten- 
uation between any CAVD composite task and any random 
sampling of all of CAVD. It would, however, be extremely 
difScult to obtain facilities to test any large group with 
samples from CAVD at all levels of difficulty. College stu- 
dents, for example, would properly rebel at being given a 
long list of absurdly easy tasks. It would discourage chil- 
dren in Grade 5 to be asked hundreds of questions none of 
which they could answer. It is not necessary to sample 
all of CAVD in the case of any one individual. If our 
sampling goes down far enough to get near his level of one 
hundred percent of successes and up far enough to get 
near his level of zero percent of successes save by chance, 
our purpose will be served. The correlation between num- 
ber right in such a sampling and number right in a com- 
plete sampling will be almost perfect. 

Our data are for such samplings. For example, a group 
of 246 pupils in Grade 9 were measured in some 200 CAVD 
tasks, sampling CAVD from level I to level K. This sam- 
pling was cut into two halves at random, save that each 
half had the same number of tasks at each level of diffi- 
culty and the same number of C, A, V, and D. The corre- 
lation of these two halves was .826 by Sheppard’s and .855 
by Pearson’s formula. The correlation of the score in the 
200 with the score in another 200 may then be taken as .90 
or .92 according to the formula used. 

565 
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The raw correlation between the number right in the 
40-composite I and the number right in the entire 200 is .72. 
The self-correlation of the former is .73. The correlation 
corrected for attenuation is .89. 

The raw correlation between the number right in the 
40-composite J and the number right in the entire 200 is .89 
by the Sheppard formula and .84 by the Pearson. The self- 
correlation of J is .80 by the Sheppard and .77 by the Pear- 
son. The correlation corrected for attenuation is 1.05 by 
the Sheppard and 1.00 by the Pearson. 

These and similar facts for tasks of various degrees of 
difficulty are shown in Tables 161 and 162, which report the 
results from this group of 246 and from three other groups 
as follows : — 

192 other students in Grade 9 were tested with a some- 


what similar long series, including the tasks of the 40-Com- 
posites K, L, M and N. 121 candidates for college entrance 
were measured by a series of 240 completions, 56 arithmetic 
tasks, 50 vocabulary tasks and 41 paragraph-reading tasks. 
Tanging from such as almost all could do to such as hardly 
any could do. A summated total score for the number 
right was computed. The tasks included the 40-Composites 
N, O, P and Q. 240 first-year students in a Law School, 
all college graduates, were measured by a series of 53 com- 
pletion tasks, 56 arithmetic tasks, 100 vocabulary tasks and 
41-paragraph-reading tasks, ranging from such as almost 
all could do to such as hardly any could do. A summated 
score for CAVD giving equal weight to C, A, V and D was 
computed. The tasks included the 40-Composites N, O, P 
and Q. 

The self-correlations of the 40 composites for the group 
of 121 high-school graduates are the averages from deter- 


minations, first by Y J -- ”* — and second by .03 + r 4 o, using 

1 ~r ^ 20 . 20 * 

only nearest neighbors among the composites. Those for 


the group of 240 college graduates are determined by the 


second method. 


38 
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The average of the seventeen corrected coefficients of 
Table 161 is 1.00 with a mean square error of ± .058. 

The average of the eight corrected coefficients of Table 
162 is 1.00^ with a mean square error of ± .034. 

These results are corroborated by results from a group 
of 100 university students, from the 240 college graduates 
of Table 162 but using different 40-composites, and from a 
group of 147 pupils in Grade 5^. 

TABLE 162. 

The Correeatiohs Between the Number op Single Tasks Bespondeb to 
Correctly in Various 40 -Composites anb the Number op Tasks Bb- 
SPONDED TO Correctly in a Long Series op CAVD Tasks Banging 
PROM Tasks Very Easy for the Group in Question to Tasks 
Very Hard for the Group in Question : Groups 13 and 17. 


Group ^ 

Compo- 

site 

Raw Cor- 
relation 
with Long 
Series 

Self Cor- 
relation 
of 40- 
Composite 

Self Cor- 
relation 
of Long 
Series 

Corrected 
Correlation 
with Long 
Series 

121 higfh -school graduates N 

.80 

.69^2 

.90 

1.01 

121 high'school graduates O 

.89 

.7SVj 

.90 

1.00 

121 hlgh-sehoo] graduates P 

.92 

.78% 

.90 

1.03 

121 high-school graduates Q 

.77 

.69 

.90 

.98 

240 college graduates N 

.85 

.86 

.95 

.94 

240 college graduates O 

.92 

.86% 

.95 

1.02 

240 college graduates P 

.89 

.79% 

.95 

1.02 

240 college graduates Q 

.86 

.72 

.95 

1.04 


One hundred students of education, graduates of col- 
leges or normal schools, were measured by 40 sentence- 
completion tasks, 4G arithmetic tasks, 150 vocabulary tasks, 
24 paragraph-reading tasks, 170 information tasks, 180 
tasks involving information plus more or less general 
sagacity, 20 picture-completion tasks, 32 pictorial analogy 
tasks and 40 geometrical relations tasks. A general sum- 
mated score, S 2 , was compiled by combining the results of 
the 40 C, 150 V and 24 D tasks so as to give a reasonable 
weight to each sort of task. Two composites (O 75^ and 
Q96i), each consisting of 26 single tasks (lOC + lOV 
-f- 6 D, the D scores being multiplied by If) of approxi- 
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mately equal difficulty within each composite and differing 
between composites so that 77 percent of the group had 15 
or more right in the O 75^ composite while 21 percent had 
15 or more right in the Q 96J composite, showed correla- 
tions of .91 and .78^ with the total summation score and a 
correlation of .66 one with' the other. 

The self-correlation of the total summation score is 
almost certainly not over .96 and the average self-correla- 
tion of the 40-composites is almost certainly not over 
.66 + .10 or .76. Using .96 and .75, the corrected correla- 
tion between one level and the total score in all levels in 
.91 + .78^ 

this group is ^ or a little under 1.00. 

V ‘90 X *76 

The 240 college graduates were measured by three com- 
posite tasks slightly different from N, O and Q, which we 
shall denote by 1, 2 and 3. Calling the long total score Si, 
the correlations are: 

*73 

3*1181 *03 

3*2i3i *73 

1*1181 = *85 
Tsiai = .91 

1*81*1 *86 

The self-correlation of 1, 2 and 3 may be set as about 
.03 higher than the correlations between 1 and 2 and be- 
tween 2 and 3, or at .76.^ 

We do not know directly what the self-correlation of 
the Si score is, but it can hardly be higher than .95. Using 
.76 and .95, the corrected correlations between score for a 
composite of tasks at one degree of difficulty and score in 
the total CAVD series are 1.00, 1.07 and 1.01, averaging 
1.03 with a mean square error of dz .031. 

One hundred and forty-four pupils at the very begin- 
ning of Grade 6 were tested with a fairly long series of 

the ease of the 240 college graduates the correlation between two 
40 composites of about equal difficulty was .73 
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CAVD, with time to do all that they could do. A summa- 
tion score was given with approximately equal weight to 
C, A, V and D. Included in the series were the 40 tasks 
of the composite I and the 40 of the composite J. The 
raw correlation between the number right in level I and 
the summation score w’as .91. The raw correlation between 
the number right in level J and the summation score was 
.86. The raw correlation between the number right in com- 
posite I and the number right in composite J was .75. 

We do not know the self-correlations in this group di- 
rectly, but that for a 40-composite will be near .78 and that 
for the total score will be near .95. The corrected coeffi- 
cients will thus here also average close to unity (using .78 
and .95, they are 1.00 and 1.06). 

We may approach the question of whether one of our 
40-composite CAVD tasks measures (except for the chance 
error due to its having only 40 tasks) all of intellect CAVD 
and nothing but intellect CAVD by another method. If it 
does, the average raw correlation rtin should in a group of 
wide range in intellect be little, if any, less than Vrtit 2 . 

We have found rtiii in groups of the range of a school 
grade to be, according to the group and the composite, .72, 
.77, .781, *791, .791, .80, .81, .811, -85, .85, .86, .86, .86, .861, 
.87, .89, .89, .891, .90, .91, .91, .91, .92 and .92.* The median 
is .86; the average is .853. 

The self-correlation rtit 2 y estimated by the Spearman 
formula from the two halves, or by taking the correlation 
with a neighboring composite -j- .03, is, for the same com- 
posites in the same groups,® .681, -69, .69, .691, .691, .711, .72, 
.75, .75, .751, .76, .76, .76, .761, .78, .78, .781, -781, .791, .791, 
.86, .86, .861 and .871. median is .76; the average is 

.765. 

s Wher6 the correlation is determined by both Sheppard and Pearson f or- 
mtilae we use the average of the two results. 

BAs before^ when both Sheppard and Pearson coefficients were obtained, 
the average of the two is used. 
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Within a grade range then rtm is .857 and rtit 2 is 
.857 

.76|. rtm is of Vrtit 2 or .98 Vrtit 2 . If the range is 

increased, rtm will approach still nearer to y/Tt^tz. 

We may then safely conclude that the ability measured 
by a sufficiently extensive composite of CAVD tasks at a 
level of difficulty which is appropriate for the group in the 
sense that the percent of successes in the group will be be- 
tween 10 and 90, is substantially identical with the ability 
measured in that group by the total CAVD series. Our 
CAVD 40-composite tasks measure intellect CAVD as far 
as their self-correlations permit. 

The facts presented here concerning the correlations 
between 40-composites and long series which approximate 
to total CAVD series corroborate the conclusion in Ap- 
pendix IV from the correlations between composites that 
intellect CAVD is to a high degree homogeneous, the 
higher levels requiring much the same ability as the lower, 
but more of it. 

There is good reason to believe that, if we had taken 
intellect GOPI, composed of tasks in observing geometrical 
relations, giving opposites, completing pictures and an- 
swering questions requiring information such as intellec- 
tual people acquire, and carried out the same sorts of in- 
vestigations as we have carried out with intellect CAVD, 
we should have found that intellect GOPI also was nearly 
or quite homogeneous at all levels of difficulty, and that a 
task composed of equal parts of G, O, P and I all of ap- 
proximately the same difficulty, measured all of intellect 
GOPI and nothing but intellect GOPI, so far as its own 
self-correlation permitted. The same would also probably 
hold for any other selection of tasks in thinking correctly 
with ideas and symbols. Nothing was done in our selec- 
tion of CAVD to favor homogeneity. There is no reason 
to believe that CAVD is any more homogeneous at differ- 
ent levels of difficulty than any other selection of tasks 
for intellect. 



APPENDIX VI 


The Estimated Fobm of Distribution of Various Groups 
We do not know exactly what the form of distribution 
of intellect in the 180 adult imbeciles is. They were se- 
lected to include the individuals 16 years old or over in two 
institutions who were from 2^ to 5 years of mental age by 
the Stanford Binet ; and on this basis we should expect the 
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distribution to be a small segment of the low end of the 
distribution of all persons 16 years old or over, but modi- 
fied by the error of measurement. Empirically the distri- 
bution of the scores in the Stanford Binet is as shown in 



Fro. 139. The result of the application of an error of measurement to the 

group represented in Fig. 138. 



Age. 


Fig. 140. By the summation of credits in the 240 C AVDIO 
tasks, the distribution was that of Fig. 141. 

In view of these facts, we have ehdsen as the probable 
form of distribution of altitude of intellect in these 180 
individuals, if they were measured in equal units, the form 
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shown in Fig. 142.' According to it the a values of the 
difficulty of the CAVD composites A, B, C, and D are re- 
spectively, — I.GSoaj “h .OSobj + l.lSoc and + 1.83od* 

If the low end of the distribution were a fraction of the 
probability surface undisturbed by the operation of any 
large factors, such as diseases, accidents at birth, and the 



in CAVDIO (Iz=Inf., 0 = 0pp.) 



Fio. 142. The probable form of distribiitioii of altitude of intellect in the 

group of 180 adult imbeciles. 


like, a section of it would be like that shown (from — 3.6o 
to — 5.0a) in Fig. 137, or from — 5.0a to — 6.0a in Fig. 
138.“ The application of an error to such a surface would 
alter it to something like Fig. 139, by blurring its distinc- 
tions and providing it with tails at both ends. 

^ We make no claim that this is the beat guess at the form of distribution 
that could be made. On the contrary, we could ourselves improve it by (a) 
giving other tests to this group, (b) by making a census of records of the 
mental ages of the inmates (16 years old or older) in institutions for the 
feeble-minded, and in other ways. We simply have not the time to do so. It 
is not a matter of much importance except in the case of the estimate of the 
difficulty of Composite D. In that case the exact determination of the upper 
tail of the distribution woiild be very desirable. However, the estimate for 
Composite D would be hazardous in any case because of the unreliability of the 
00.6 as the percent of successes. 

2 The ordinate scale of Fig. 138 is magnified 100 times that of Fig. 137. 
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We know very little, however, about the forces operat- 
ing to produce individuals so dull as these; or the form 
which the extreme low end of the distribution of human 
intellect takes. 



Fro. 143. A section of the normal surface of frequency from —4.2 to -3.7. 

We do not know exactly the form of distribution of the 
100 feeble-minded of Stanford mental age 6 yr., 0 mo. to 
6 yr. 11 mo. If they were a random Sampling of all per- 
sons sixteen years old or older of mental age 6, the dis- 
tribution would be approximately that of a segment of the 
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surface of frequency for adult intellect from an I.Q, of 
37^ to an LQ. of 44.4, or from — 6.25a to — 5.56a, if we 
use Terman's estimate of the variability of intelligence 
quotients [’16 p. 78]. However, the variability of the in- 
tellects of persons chronologically 16 or over is probably 
much greater than that given by Terman’s figures for chil- 
dren, at least from the mode toward the low end ; and from 

— 4.2o to — 3.70 seems a more probable status by random 
selection. 

As has been stated, we do not know the exact form of the 
surface of frequency for adult intellect, at its low extreme. 
It almost certainly is continuous from M.A. 6-11, to M.A. 
6-0, and diminishing somewhat in area per unit of abscissa. 
The low extreme of Form A from — 5.0a to — 3.6o was 
shown in Fig. 137. That from — 6.0o to — S.Oo was shown 
in Fig. 138. Not much weight should be attached to any 
estimate from theory of the amount of the diminution in 
frequency as we go to very dull levels of the total adult 
population, since we know very little about the causes 
which are acting to create these levels. Using — 4.2a to 

— 3.7a of Form A, we should have the distribution shown 
in Fig. 143. 

However, the selection for commitment is not random, 
the duller ones being more often committed than the 
brighter. So the pitch of the curve would be expected to 
be less sharp than that of the general adult population, 
whatever that may be. 

Turning to the actual measurements, the form of dis- 
tribution of the 100 imbeciles of mental age 6, if we as- 
sume that one month of mental age from 6 to 7 by that scale 
equals any other month, and that the Stanford Binet mea- 
sures intellect perfectly, is that shown in Fig. 144. 

We do not know what the values of these mental months 
of the StajUf ord Binet are in truly equal units, as there have 
never been any measurements of grade or age groups by 
the Stanford Binet which are large enough to enable us 
to apply the procedure which we have used with the Na- 
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tional, Otis, Tenuan group and other examinations. In 
all probabihty, the diiferences in the true values are not 
great. 

We know that the Stanford Binet does not measure in- 
tellect perfectly; and we know roughly the amount of the 
error of a single determination of the Binet as a measure- 
ment of the abilities that would be measured by a long 
series of examinations of the very same sort as the Binet. 



52 56 60 64 65 72 76 60 6 4 68 92 96 100 104 

FiO. 144. The form of distribution of Group im. 6 in Stanford -Binet Mental 

Age. 

This is a mean Ot-o of about 6 mental months, at or near 
mental age 10. It probably is less around mental age 6, 
but it will still be large in comparison with the range of 
12 months in the measurements themselves. Taking it 
as 4 mental months, the true intellects of the 100 would 
range from about 5 yr. 0 mo. of mental age to about 8 years 
of mental age. However, a long series of examinations of 
the very same sort as the Binet would probably not mea- 
sure aU of intellect. So that an allowance for the error of 
the Binet sort of examination, no matter how extensive, 
must also be made. 

It is thus very difficult to make anything like a valid 
estimate of the probable form of distribution of altitude 
of intellect measured without error and in equal, units. 

What we have done is to apply an^ error of the magni- 
tude shown in Table 163 to the empirically obtained scores 
grouped as: 
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The 50 cases of feeble-minded of Class 3 may be as- 
sumed to be of approximately the normal form of distribu- 
tion in respect of intellect, since they represent those se- 
lected by the educational authorities of the institution as 
belonging in Grade 3 rather than in Grade 2 or Grade 4. 
We have seen that the process of educational selection for 
a grade tends in general to produce symmetry and an ap- 
proximation to Form A. However, there would probably 
be a curtailment at the upper end and an extension at the 
lower, since there would be a scarcity of children who were 
much too bright really to belong to Grade 3, but an abun- 
dance of children much too dull to belong there. Whereas, 
in an ordinary school the forces acting to produce grada- 
tion in Grade 3 select from a rectangle, in an institution 
for feeble-minded they select from a surface which is pre- 
sumably much higher at the low than at the high end. 

In a total CAVD summation score, the distribution was 
that shown below. 


Quantity 

Frequency 

60 to 

79 

I 

80 

99 

0 

100 

119 

1 

120 “ 

139 

1 

140 

159 

0 

160 ‘‘ 

179 

3 

180 “ 

199 

4 

200 

219 

7 

220 

239 

13 

240 << 

259 

7 

260 “ 

279 

7 

280 

299 

4 

300 

319 

2 


In view of these facts we have assumed the form of dis- 
tribution of this group to be that shown in Table 164. 
Using this, the difficulty of Composite E is — 1.35cte ; that 
of Composite F is — 1.250?; that of G is — .SSoa; that of 
H is — .41aH ; that of I is + 1.17oi. 

It has not been practicable to secure sufficient informa- 
tion outside of our own tests for even the roughest em- 
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pirical determination of the form of distribution of the 
Special-Class group. Nor is enough known about the 
policies of the different schools, principals and teachers, 
nor about the accuracy of the diagnoses to justify an 
a priori estimate of the selective forces which relegated 
these pupils to the special classes. There should theoreti- 
cally be much of negative skewness, since the selection is 
surely from the dull half and probably from the dullest 

TABLE 164. 

Fori£ of Distribtjtion Assumed in Obtainino Measures of the Difficui/ty 
OF Various Gomfosites fob the Group of 50 Feeble-Minded. 


Interval 


Frequency 


a 

to 

a + 

k 

1 

a-f k 

( t 

a + 

2k 

1 

a-l- 2k 

i < 

a + 

3k 

2 

a-l- 3k 

i € 

a + 

4k 

2 

B+ 4k 

tt 

a + 

5k 

3 

B+ 5k 

i ( 

a + 

6k 

3 

a+ 6k 

t < 

a + 

7k 

10 

a-h 7k 

i i 

a + 

8k 

12 

a + 8k 

a 

a + 

9k 

20 

B-^ 9k 

n 

a + 10k 

20 

B + 10k 

1 1 

a+llk 

12 

a + llk 


a+12k 

10 

B+ 12k 

n 

a+ 13k 

2 

a + 13k 

ft 

a + 14k 

2 


quarter or eighth rather than from the brighter which 
results if the effort is to select the fraction of the 
normal surface between — 1.5 S.D. and — 2.7 S.D., but this 
effort is disturbed by an error distributed as follows, — .2 
S.D., 1 ; — .1 S.D., 4 ,* 0 S.D., 6 ; -f .1 S.D., 4 ; + .2 S.D., 1. 

This gives a surface of the form shown in Fig. 146. 
Using this form of distribution, the differences in difficulty 
of E, F, G, H, and I, from the median difficulty for the 
Special Class group are respectively, in units of the a of 
the group, — 2.61a^, — 2.31<tf, — 1.54ao, — .44ffH and 
-f- .36cr,. 




Fla. 147. The form of distribution of Grade 5 in Army Alpha^ in equal units. 



Fza. 148. The form of distribution of Grade 5 in Examination A, in equal 

units. 





Pio. 149. The form of distribution of Grade 5 in the National Intelligence Exammatu 
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To ascertain the form of distribution in Grade 5 and 
the form of distribution in Grade 4, we have collected the 
distributions of scores made by these grades in any of the 
examinations whose scores we have transmuted into scales 



t » - < - I - I I 1 I , I ■ - t- - — rrr: 

Fig. 152. The fom of distribution of Grade 4 in National A, in equal units. 

in equal units. The surfaces of frequency are drawn, using 
the equal-unit scales.- Figures 147, 148, 149 and 150 show 
the essentials in the case of Grade 5; Figures 151, 152, and 
153 show the essentials in the case of Grade 4. 
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Fio. 153. The fona of distribution of Or ado 4 in Otis Ezaminatioiiy in 

equal units. 

In view of these curves and the general probabilities of 
the case, it does not appear that there is justification for 
assuming any considerable flattening or any considerable 
skewness. So the form of distribution for Grade 5 and 
for Grade 4 is taken as that of the normal probability sur- 
face. 

In order to perfect our measures of the differences in 
difficulty of levels N, 0, P and Q, we need knowledge of 



Fig. 154. The form of distribution of first-year law students, '24, in Army 

Alpha iu equal iiuitB. 
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the form of distribution of altitude of intellect in the group 
of 240 college graduates whose i>ercentage of success at 
each of these levels we have computed. This group of 240 
comprised the first-year class entering the Columbia Law 
School in 1924. We have their scores in Army Alpha taken 
on the same day that they took the CAVD examination; 
and we have learned (in Chapter VII) the approximate 
values of Army Alpha scores in truly equal units. The 
distribution of the 240 in ^Ipha in equal units is shown in 
Fig. 154. The Law School class entering in 1924 may be 
regarded as differing from the classes entering in 1921, 
1922, and 1923, only by chance in respect of the form of 
distribution. In the ease of each of these classes we have 
the scores in several intelligence examinations, namely: 

CLASS OF 1921 

A completion test of about 1 hour. 

A paragraph reading test of 1 hour. 

A paragraph reading test of 1 hour (selective type). 

Thorndike Exam., series 1925-1930, Part I, Form Q. 

Thorndike Exam., series 1925-1930, Part I, Form R or S. 

An arithmetical composite. 

GLASS OF 1922 

A paragraph reading test of 1 hour. 

A paragraph reading test of 1 hour (selective type). 

An examination containing opposites, Briggs’s gram- 
matical analogies and a vocabulary test 

Thorndike Exam., series 1925-1930, Part I, Form S or V 

GLASS OF 1923 

An extensive test in sentence completion and arithmeti- 
cal problems. 

An examination containing opposites, Briggs’s analo- 
gies, and a vocabulary test. 

A paragraph reading test of 1 hour. 

These records permit the application to “a first-year 
class in the Columbia Law School” of the same processes 
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of reasoning nsed in Appendix III in the case of “a grade 
popnlation.*' The surfaces of frequency for each sepa- 
rate score are shown in Figs, 155 to 167. They show in 




-2 -I -O +l +2 

Fiq. 158. Form of distribution of first-year law students, '21, Thorndike, 

Part I-Q. 
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Fia. 159. Form of diatribation of first-year lavr students, ’21, Thorndike, 

Part I, B or S. 



Fig. 160. Form of distribution of first-year law students, ’21, Arithmetic. 
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general a departure from ** normality** in the shape of 
a longer tail at the low end. This is still clearer when we 
rid the measurements of chance variations by combining 




THE FOBM OF DISTBTBITTIOlSr OF INTELiIjEOT 


591 




S or IT. 


592 


THE MEASUBBMEmD OB INTBLlilGBEOE 


the scores for each individual in the case of the 1921 and 
1922 groups,* as shown in Figs. 168 and 169. When the 
surfaces of frequency are combined (with equal weight for 
each examination) for each class, we have Figs. 170, 171, 
and 172. When these are combined (with equal weight to 
1921, 1922, and 1923) we have Fig. 173. 



Fig, 165. Form of distribution of first-year law students^ '23, Co. and Ar* 

Both the direct evidence from the 1924 group itself, 
when measured by Alpha transposed to a scale of equal 
units, and the evidence from the groups of 1921, 1922, and 
1923 in which the inequalities of units may be assumed to 
have approximately counterbalanced one another, show a 
negative skewness. So also does the distribution of the 
1923 groujj when measured by Army Alpha with a scale of 
equal units. This is shown in Fig. 175. So we have taken 

8 This would be true also with the 1923 group, but we have not had time 
to make the computations exactly. 
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-,3 -2 +1 -0 +1 

Fio. 171. Composite of the distributioiis of iFigs. 161 to 164. 
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Fiq, IT3. Composite of the carves of Figs. 170, 171 and 172, with equal 

weight allowed to each. 
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the distribution of Table 165, which is that of Fig. 173 with 
some smoothing, as representing the probable form of dis> 
tribution of altitude of intellect in the group of 240. 


TABLE 165. 

The PbobabiiB Form of Distbibittion of Ai<titdoe of IsTtJJuacT isr Gbouf 

17 (liAW Students). 


Interval 

Frequency 

permille 

Interval 

Frequency 

permille 

L 

to 

L+ 1 

i 

L + 33 

L + 34 

87 

L+ 1 

( < 

L+ 2 

i 

L + 34 

“ L + 35 

38 

Ii+ 2 

f i 

Li+ 3 

i 

L + 35 

L + 36 

39 

Ii+ 3 

t4 

Li + 4 

i 

L + 36 

'' L + 37 

39* 

L+ 4 

4 i 

L+ 5 

1 

L + 37 

“ L + 38 

39* 

Ij+ 5 

4 4 

Ii+ 6 

1 

L + 38 

'' L + 39 

39* 

L.4- 6 

4 4 

Li+ 7 

li 

Ii + 39 

“ L + 40 

39* 

^4- 7 

4 4 

Ii+ 8 

li 

L + 40 

L + 41 

39 

L+ 8 

i i 

Li+ 9 

2 

L + 41 

“ L + 42 

38 

L f 9 

4 4 

Li + 10 

2 

L4 42 

L + 43 

37 

Ii + 10 

44 

L + 11 

2i 

L,-h43 

‘‘ L + 44 

36 

Li + 11 

4 4 

Ii+12 

2i 

LH-44 

L + 45 

34* 

Ii+12 

4 4 

Li +13 

2i 

L4 45 

“ L + 46 

33 

Li + 13 

4 4 

L+ 14 

3 

L + 46 

L + 47 

31 

L. + 14 

4 4 

L + 15 

3i 

L. + 47 

L + 48 

29 

L + 15 

4 4 

L+16 

4 

L+48 

L + 49 

25* 

Lt + 16 

44 

L + 17 

4i 

L + 49 

“ L + 50 

22 

Ii + 17 

4 4 

Ii + 18 

5i 

L 4 50 

“ L + 51 

19 

L + 18 

4 4 

L + 19 

6 

L + 51 

L + 52 

16 

L. + 19 

4 4 

Li + 20 

7 

L + 52 

“ L + 58 

13* 

Li + 20 

4 4 

L + 21 

8 

L + 53 

Lf54 

11 

L + 21 

4 4 

L + 22 

9 

L + 54 

“ L + 55 

8* 

Li + 22 

4 4 

L + 23 

10 

L + 55 

“ L+56 

6 

11 + 23 

4 4 

L + 24 

12 

L + 56 

“ L + 57 

4 

L + 24 

44 

L + 25 

14 

L + 57 

L + 58 

3* 

Li + 25 

4 4 

L + 26 

16 

L + 58 

L + 59 

3 

Li + 26 

4 4 

L + 27 

18* 

L + 59 

L + 60 

2* 

L + 27 

4 4 

L + 28 

21 

L + 60 

“ L + 61 

1* 

Ij + 28 

44 

L + 29 

24 

L + 61 

L + 62 

1* 

Ij + 29 

4 4 

L + 30 

27 

L + 62 

“ L + 63 

1 

Li + 30 

4 4 

L + 31 

30 

L + 63 

“ L + 64 

1 

Ii + 31 

4 4 

L + 32 

33 

L + 64 

^‘•L + 65 

* 

L + 32 

4 4 

L + 33 

35* 

L + 65 

“ L + 66 

* 
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The percents correct for the four levels in question 
being 95.4, 77.1, 56.7, and 22.9, the values in terms of 0 dis- 
tances from the median, are — 1.862, — .714, — .153, and 
+ .738 in terms of On, Oo, Op and Oq, respectively. 




in equal units. 


In the case of the 44 recruits, a normal form of distri- 
bution is assumed because nothing demonstrably better is 
suggested by the facts available. These facts are: the re- 
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ported amount of schooling, the distributions of scores for 
the group in Otis Intermediate, G.E. Form I (a General 
Electrical Information test) and the sum of the scores in 
Composites I, J, and K (but with the arithmetic of K esti- 
mated as They appear in Table 166. 

TABLE 166. 

Data fob Estiacatiko the Fobu; of Distbibotioh of Altitude of Intellect 

IN THE Oboup Ad. (44 Becrutts). 


Grade Beached 
Grade Preq. 

Otis 

Score 

Int. 

Freq. 

G. 

Score 

E. 

Freq. 

1 + J + K 
Score Freq. 

3 or 4 

1 

15-19 

4 

3- 5 

3 

10-29 

2 

5 or 6 

10 

20-34 

10 

6- 9 

6 

30-49 

9 

7 or 8 

18 

35-49 

13 

10-13 

26 

50-69 

0 

9 or 10 

11 

50-64 

12 

14-17 

8 

70-89 

11 

11 or 12 

3 

65-79 

4 

18-21 

2 

90-109 

10 





22-25 

0 

110-129 

3 





26-29 

1 







30-33 

0 







34-37 

0 







38-41 

1 








LIST OF REFERENCES 


Anderson, J. E. ’20 
Bailor, E. M. ’24 . 


Baldwin, B. T., and 
Steeher, L. E. ’22 

Binet, A. ’16 

Boas, F., and 
Wissler, C. ’06 


Brooks, F. D. ’21 


Intelligence Tests of Yale Freshmen. School and 
Society 11: 417-420. 

Content and Form in Tests of Intelligence. Tcach- 
eis College, Columbia University, Contributions 
to Education, No. 162, 1924. 

Afental Growth Curve of Normal and Superior 
Children. University of Iowa Studies 2: No. 1. 

The Develojiment of Intelligence. (English trans- 
lation by Kite.) 

Statistics of Oroivth. Report of the IT. S. Com- 
missioner of Education for Year ending June 
30, 1904, 1: 25-132. 

Changes in Mental Traits with Age, determined by 
Annual Re-tests. Teachers College, Columbia 
University, Contributions to Education, No. 116, 
1921. 


Bulletin, July 21, 1923 Bureau of Educational Reseaich, Universitv of 

North Carolina. 

Burt, C. ’21 Mental and Scholastic Tests. 

Carey, G. L., and 

ECline, L. W. ’23 The Ivluie-Carey Measuring Scale for Free-hand 


Clark, J. R. ’25 
Cobb, M. V. ’22 


Colvin, S. S. ’22 

Colvin, S. S., and 
MacPhail, A. H. ’22a 

Colvin, S. S., and 
MacPhail, A. H. ’24 


Coxe, W. W. *21 


Drawing. The Johns Hopkins University 
Studies in Education, No. 5a, 1923. 

The Relation of Sliced, Range, and Level to 
Scores on Intelligence Tests. 

The Limits Set to Educational Achievement by 
Limited Intelligence. Jrur. Educational Psy. 
13: 449-464 and 546-55^ 

The Present Status of Mental Testing. Educa- 
tional Review 64: 196-206 and 320-327. 

The Value of Psychologi<*al Tests at Brown Uni- 
versity. School and Society 16: 113-122. 

The Intelligence of Seniors in the High Schools 
of Massachusetts. Dept, of the Interior, Bureau 
of Education, Bulletin No. 9, 1924. 

Variation in General Intellig**nee. Jour, of Edu- 
cational Research 4: 187-194. 


40 


601 



602 


LIST OF REFEBENCES 


Haggerty, M. E. *23 Haggerty Intelligence Examination; Manual of 

Directions. 

Hammarberg, Carl ’93.-Studier ofver Idiotiens Elinik och Patologi. 

Hillegas, M. B. *12 Scale for the Measurement of Quality in English 

Composition by Young People. Teachers Col- 
lege Record 13; 331-384. 

Hbllingworth, L. S. ’23_.The Psychology of Subnormal Children. 

Kelley, T. L. *19 The Measurement of Overlapping. Jour. Educa- 

tional Psy. 10 : 458—461. 

Kelley, T. L. *21 The Reliability of Test Scores. Jour. Educational 

Research 3 : 370-379. 

Kelley, T. L. '23 Statistical Method. 

Kelley, T. L. *23a The Principles and Technique of Mental Measure- 

ment. American Jour, of Psy. 34: 408-432. 

Madsen, J. N. '22 Intelligence as a Factor in School Progress. 

School and Society 15 : 28511. 

Memoirs '21 National Academy of Sciences. Psychological Ex- 

amining in the U. S. Army, edited by Robert M. 
Yerkes 15: 1921. 

Miller, W. S. '22 Miller Mental Ability Test. Manual of Directions, 

First Revision for Forms A and B. 

Myers, C. E., and 

Myera, G. C. '21. Measuring Minds. 

Myers, C. E., 

Myers, G. C., and 

Layton, S. H. '21 Group Mental Testing in Altoona, Pa. School and 

Society 13 : 624-628. 

Noble, E. L., and 

Arps, G. F. '20 University Students’ Intelligence Ratings Accord- 

ing to the Army Alpha Test. School and So- 
ciety 11: 233-237. 

Otis, A. S. '21 — Otis Group Intelligence Seale; Manual of Direc- 

tions, 1921 Revision. 

Pearson, K. '14 Tables for Statisticians and Biomctricians. 

Pressey, S. L. '20 A Brief Group Scale of Intelligence for Use in 

School Surveys. Jour. Educational Psy. 11: 
89-100. 

Ruch, G. M. '23... _.A Mental Educational Survey. Univ. of Iowa 

Studies in Education 2 : No. 5. 

The Proof and Measurement of Association be- 
* tween Two Things. American Jour, of Psy. 15; 
72-101. 


Spearman, C- *94 



LIST OF REFEKENOBS 


@03 


Spearman, G. *07 Demonstration of Formulae for True Measure- 

ment of Correlation. American Jour, of Pay. 
18: 161-169. 

Spearman, C. *10 Correlation Calculated from Faulty Data. British 

Jour, of Psy. 3 : 271—295. 

Spearman, C. *13 Correlations of Sums and Differences. British 

Jour, of Psy. 5 : 417—426. 

Spearman, C. *23 The Nature of Intelligence and the Principles of 

Cognition. 

Taylor, Q. A. *23 An Inventory of the Minds of Individuals of Si^ 

and Seven Years Mental Age. Teachers College, 
Columbia University, Contributions to Educa- 
tion, No. 134, 1923. 

Terman, L. M. *16 The Measurement of Intelligence. 

Terman, L. M. *22 The Great Conspiracy or The Impulse Imperious 

of Intelligence Tests, Psychoanalyzed and Ex- 
posed by Mr. Lippmann. The New Republic 
33: llfr-120. 

Terman, L, M. *23~ Terman Group Test of Mental Ability for Grades 

7 to 12; Manual of Directions. 

Thomson, Gk>df rey H. 

*24 -A Formula to Correct for the Effect of Errors of 

Measurement on the Correlation of Initial 
Values with Gains. Jour, of Experimental Psy. 
3: 321-324. 

Thorndike, E. L. *10 Handwriting. Teachers College Record, II, No. 2. 

Thorndike, E. L. *13 The Measurement of Achievement in Drawing. 

Teachers College Record 14: 1-39. 

Thorndike, E. Li. *13 a An Introduction to the Theory of Mental and 

Social Measurements. 

Thorndike, E. L. *16 Tests of Aesthetic Appreciation. Jour. Educa- 
tional Psy. 7 : 509-622. 

Thorndike, E. L. ’23._. On the Improvement in Intelligence Scores from 

Fourteen to Eighteen. Jour, Educational Pay, 
14: 513-616. 

Thorndike, E. L, ’24__The Influence of the Chance Imperfections of 

Measures upon the Relation of Initial Score to 
Gain or Loss. Jour. Experimental Psy. 3 : 
225-232. 

Thorndike, E. L., 

Br^gman, E. O., and 

Cobb, M. V. *24 The Selection of Tasks of Equal Difficulty by a 

Consensus of Opinion. Jour. Educational Re- 
search 9: 133-139. 



604 


LIST OF KEFEFENCES 


Thorndike, E. L., and 
Bregtnan, E. O. '24 

Thnrstone, L. L. ^21 

Trabue, M. R. ’16 

0 

Vlcieent, Leona M. ’24 
Whipple, G. M. ’22 
Whipple, G. M. ’23 
Yerkes, R. M. ’20 


On the Form of Distribution of Intellect in the 
Ninth Grade. Jour. Educational Research 10: 
271-278. 

A Cycle-omnibus Intelligence Test for College 
Students. Jour. Educational Research 4: 265— 
278. 

Completion-Test Language Scales. Teachers Col- 
lege, Columbia Univcrsit 3 ^, Contributions to Edu- 
cation, No. 77. 

A Study of Intelligence Test Elements. Teachers 
College, Columbia University, Contributions to 
Education, No. 152, 1924. 

Intelligence Tests in Colleges and Universities. 
The 21st Yearbook of the National Society for 
the Study of Education, Chap. X: 262. 

The Intelligence Testing Piogram and its Objec- 
tois — Conscientious and Otherwise. School and 
Society 17 : 561—568 and .596—604. 

P^-ychological Examining in the United States 
Array. Memoirs, National Academy of Sciences, 
15. 



INDEX 


Abilities measurable bj I. £3. B. tech- 
nique, 476 

Ability, defined, 484; to deal with 
persona, things, ideas, by ideas, 
413; to learn as criterion intel- 
ligence, 17; to learn as intelligence 
test, 408ff,; w* variability, 43ff», 
56, 497ff. 

Absolute vs. proportional counts of 
tasks, 383 f. 

Absolute zero, 294, 336f., 339ff., 471f., 
482, 485f.; and Binet Scale, 403 

Acceleration of level with age, nega- 
tive, 467, 475 

Accident, effect on form of distribu- 
tion of intelligence, 272 

Acquired vs. original ability, 95f., 
433ff. 

Adequacy of one CAVB level for 
measurement, 565 fif. 

Advantages from location of zero 
point, 339 

Ago changes in altitude and area, 
4()3fif. ; in specialized abilities, 468; 
grouping for intellectual diflS.eulty, 
28 

Alio wane o for environmental effect on 
intelligence, 460 ff. 

Altitude, as related to width, 376; 
and nature vs. nurture, 458 ; and 
speed of learning, 409; and stock 
examination score, 403f., 405f. ; 

curve with age, 463 ; measured 
without error, 577; of intellect, 24, 
33, 104, 469 

Ambiguity in content of tests. Iff. 

Analogies test, 9 

Analysis of measurement into level, 
width, speed, 35 

Anatomical cause of intellect, 420 

Anderson, J. E., 539, 541 

Annoyers, 489f. 

Applications of I. E. B. technique to 
measurement of human abilities, 
476fC. 


Arbitrariness in units, 1, BA ; 

choice of tasks as intellectual, 91 
Arbitrary scores transmuted to m^e 
variabilities equal, 520 
Area of intellect, 24, 339, 378£r«/469; 
and nature vs. nurture, 456; and 
stock examination score, 404; as 
related to altitude and width, 378 
Arithmetic problems, as tests, 
composites, construction of, 

Army Alpha, 1, 2, 3, 10, 16, 22f., 

42, 44, 46, 49, 52, 55, 99, 227, 307, 
403, 404, 410, 433, 437, 461, 475, 
491, 494, 522, 528, 534, 539, 562, 
586, 592 ; form of distribution, 

291f . ; letter ratings, 7 ; scores and 
officeis^ ratings, 405; scores in 
equal units, 228 ff., 309, 316 
Army Beta, 15, 38, 222, 433, 437, 506; 

form of dUtribution, 291 f. 

Army Examination a, 228, 475, 514, 
522, 528, 562; form of distribution, 
291f. ; scores in equal units, 264fif., 
309, 313, 316 
Ari^S, G. P., 539 

Array, form of distribution of, to 
measure difficulty, 40, 54fF. ; in cor- 
relation table to measure ability vs. 
variability, 505, 520 
Arrays used to measure difficulty, 40 
Associative vs. selective and generaliz-^ 
ing thought, 414f. 

Assumptions in measuring intellectual 
difficulty, 38, 59 

Attenuation, formula, 60, 111, 177, 
560f. ; formulae used to find com- 
mon part of tests, 406f., 429£.; 
corrected for in correlations, 424f.; 
in composites with few tasks, 556; 
in variability vs. ability, 600, 502, 
506; used on correlations of Burt, 
455f.^ 

Average ability of individual, 493f« 
Axioms of mental measurement re 
original and acquired ability, 436 
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BAGtiSTi W. C., 433 
Bailor, E. M., 135, 317 
BaXiDwik, T., 495 
BatterieB of tests, 1, 9 
Beauty of design, scales for, 139 
Beta, Army, 15, 18, 222, 433, 437, 
506; form of distribution, 29 If. 
Binrt, a., 16, 409 

Binet, mental age, 3, 6, 16, 17, 97, 
402, 447, 575; scale, zero on, 403; 
tests, 9, IS, 23, 338, 433, 435, 451, 
.454, 455, 457, 577 
Biserial r, 118, 120ff., 131, 389 
Blurring of variability w. ability by 
errors of measurement, 500, 502 
Boas, Franz, 112 

Brroman, £. O., 135, 138, 306, 521 
Briggs Analogies Test, 586 
Brigham, C. C., 540, 546 
Brooks, F. D., 466f. 

Brown University, 540; Psychological 
Examination, 228, 534, 540; scores 
in equal units, 260f?., 316 
BuChHoSo, intellect and age, 468 
Buckingham Spelling Scale, 119 
Bttrt, C., 9, 435, 447ff. 

Burt Reasoning Tests, 447ff. 

Capacities independent of intellect, 
431 

Carisv, G. L., 134 
Carpentry, 476, 480f^. 

Cattbll, J. McK., 1 
Causation of, ability, 95; area and 
altitude of intellect, 397f^., 402f.; 
associative and higher thought proc- 
esses, 425, 430; form of distribution 
of intellect, 271; intellect, 420; 
variation, 271 

CAVD, altitude, 197ff., 102 ; and other 
tests, 97fE.; composite tasks, 66ff., 
160fP.; composites, construction of, 
176ff.; correlation with other testa, 
96ff. ; intellect, 65jBP., 96ff., 10 Iff., 
104ff., 114; arid original capacity, 
435ff.; and training, 435ff.; level, 
form of distribution, 285ff. ; os. 
width, 388f.; selection of tasks, 
410; width and altitude, 102 


Central tendency as reference poin^ 
42, 43, 57, 109, 148, 209ff., 224ff., 
297ff., 314ff., 339, 491ff., 497ff. 
Ghambrrlain, a. F., 9 
Chance error, in original measures 
affecting partial correlations, 454f.; 
in stock examination scores, 411 ; 
smoothing for, 299; in variations, 
eliminated, 590ff. 

Change from 100 percent to 0 percent 
correct, 374f. 

Changes in altitude and width with 
age, 463ff. 

Cheating, 476ff., 564 
Clark, J. B., 97f., 106, 318, 405ff. 
Coaching for tests, 437 
Cobb, M. V., 135, 528, 534 
Colvin, S. S., 9, 433, 534 
Combining novelty and familiarity in 
test tasks, 439f. 

Commensurable oi ’s in various groups, 
303ff. 

Common causation of associative and 
higher thought processes, 425, 430 
Common measure for intellect of ani- 
mals and men, 840 
Common reference point for difficulty 
of composites, 314ff., 331f. 

Common sense, criteria for test items, 
64; test, 9 

Comparative, enumeration of ideas, 
419; variabilitiea in grade groups, 
311f . ; in face-value and equal 
units, 318ff. 

Comparison of, consensus with experi- 
mental results, 143 ff.; variabilities, 
303ff. 

Cbmpletion, composites, 193ff.; tests, 
9 (See also Trabue Completion 
Tests) 

Composite ta^s, as tests, 118f., 128, 
133 ; vs, single tadis as measures of 
intellect, 470 

Composites, A to I, 294; difficulty of, 
295ff.; I to Q, 294; of specided 
difficulty, 128f.; made by consensus 
ratings, value of, 156ff. 

Composition, of word knowledge lev- 
els, l&4f.; scales, 134, 476ff« 
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Coneept of altitude and area, 379 

Connection between ideas and sequent 
ideas, 417 

Connection^fonning vs* higher thought 
processes, 414fl. 

Connections to vaxylng ideas from one 
idea, 419 

Consensus of expert opinion as mea- 
sure of intellectual difficulty, 134ff., 
156ff.; of experts for location of 
zero point, 342ffi ; ratings, under 
and over estimation in, 157; valid- 
ity of, 141 

Consequences from given stimuli, 420 

Considerations limiting test Items, 
63f. 

Constant error, in Judging, affecting 
results, 134 ; in intellectual diffi- 
culty of tests, 141ff., 145, 148ff., 
155f»; in scoring affecting results, 
132; tending to symmetry, 271f. 

Constitution of intellect, 50 

Construction, of CAD composites, 
193ff.; of V composites, 179ff. 

Content of intelligence tests, 20f. 

Conventional zeros, 487f, 

Correction for, attenuation, 392f., 
424ff., 567 ; chance, 299, 301 ; 

range, formula for, 155; remoteness 
of level, 298f. ; restricted range, 
155, 556, 562 ; sampling error, 112f., 
118 

Correlation, as tool for discovering 
differences in tasks at same levels, 
382 ; of associative and higher 
abilities, 42 f.; of GAVD level and 
CAVD score, 105f.; of CAVD level 
and other test scores, 105ff . ; of 
CAVD score and other teat scores, 
97ff.; of CAVD score and Stanford 
M.A., 97 ; of CAVD width and alti- 
tude, 102 ; of CAVD with other 
tests, 96ff.; of C and A tasks, 
115 ff.; of composites, 376; of com- 
posites more and less remote, 557; 
of consensus and experimental re- 
sults, 143 ff.; of measure of altitude 
and success-failure in composites, 


389ff., 394ff.; of officers^ ratings 
and Army Alpha scores, 405; of 
one composite and summation score, 
302; of one level and sampling of 
entire series, 565ff.; of single tasks 
and inteUect, llSff., 129ff.; of sub^ 
series in CAVD with total CAVD, 
102ff.; of tasks of one sort and 
composites, 154; tetrachoric, 390; 
technique vs* line of best flt, 339f.; 
technique vs* probable error of 
measurement, 520 * 

Correlations corrected, by Spearman- 
Brown formula, 299ff., 324, 426, 
559, 561 ; for attenuation, 406f., 
426ff., 455, 560, 561, 562, 570f.; 
for range, 155, 557ff., 562, 570f.| 
for remoteness of levd, 563 
Correlation, self-, of ratings, 138; of 
sub-series in CAVD, 107; of task, 
50; of various tests, 99 ff. 
Correspondence of degrees of CAVD 
and other variables, 103 
Courtis tests in computation, 492 
COXE, W. W., 522 
Credits, graded, 29f., 477, 480 
Criteria for, choice of test items, 64; 
composites, 129f • ; estimating na- 
ture vs* nurture, 441f.; intellectual 
difficulty, 63f . ; intellectual tasks, 
60f. ; intelligence, 10, 15 ff.; used in 
making N. I. T., 409f.; value of 
mental product, 14 

Curve, Gaussian, 6, 53, 270, 521, 527, 
536; of altitude and age, 463S.; of 
area and age, 467ff.; of percent 
correct, 351 ff. 

Data for intelligence testing, 20f. 
Dearborn Intelligence Examination 
41, 493 

Definition of, ability, 484; idea, 418; 
intellect, 25f., 27, 412; intellectual 
difficulty, 26f . ; product, 25ff . ; 
range, 26f.; task, 26f.; one idea, 
418f. • 

Democracy and the I. Q., 433 
Derivation of units of measure, 134 
Design, scale for beauty of, 134 
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Determinism^ educational, 433 

Development with age as criterion of 
intelligence, 16 

Differences in CAVD composites N to 
Q, 304ff«, 320f.; consensus and ex- 
perimental results, 142 ff.; difficulty 
of composites, 485; from common 
reference point, 415f., 420f. ; grade 
medians in terms of 0 ,^, 218f. ; 
individuals' connection systems, 
314ff., 331ff.; moaning of scores in 

• stock examination and CAVD level 
score, 407; original capacity, 421f.; 
0j by two methods of calculation, 
303 ; tasks at the same level, 380ff . ; 
diagnosis of nature and nurture, 
460 

Differential gains in intellect with 
ability, 288; age, 287ff. ; training, 
288f«; vocational selc^ction, 289 

Difficulty, CAVD, 103; in general 
intellectual difficulty, 28, 56f., 119, 
126f., 129, 133 ; intellectual, 22f«, 
26, 28, 62ff., 109ff. ; criteria for, 
63f.; of a task, 109; of CAVD 
composites, 210ff.; A to H, 321ff.; 
A to Q, 471; I to Q, 295ff.; related 
to percent correct, 351ff*; fiom zero 
as reference, 349f*; of CAVD lev- 
els, 179; of tasks measured by per- 
cent successful, 39, 62, 109, 121, 
123ff., 144f., 148ff., 185f., 190ff., 
195ff., 202ff., 205ff.; of word knowl- 
edge tasks, 182f. 

Directions, composites, construction 
of, 193ff.; test, 9 

Disciplinary value of school studies, 
440 

Discrimination of pitch, 47 6f., 486 

Disease, effect on form of distribution 
of intellect, 272 

Dissociation integration, 431 

Distribution, form of (Sec Form of 
Distribution) ; Symmetry of, 270f. 

Drawing scalea/ 134, 476ff, • 

Errors of measurement, 58, 132; due 
to carelessness or cheating, 564; in 


stock examination scores, 411; re- 
duced by successive summation of 
scores, 530, 545ff. ; tending to spuri- 
ous normality, 271, 529, 545, 550, 
572ff., 577ff., 590ff.; vs. initiri 
ability and gain, 408f . 

Errors of sampling, 109f., 112, 114, 
117f. ; as affecting partial correla- 
tion coefficients, 454ff. 

Estimated effect on correlation of 
small composites, small range of 
age and levels, 556 

Estimating CAVD altitude of individ- 
ual, 369 

Examinations, school, as tests, 1 

Experiment program to locate zero, 
339ff. 

Expert opinion as measure of intel- 
lectual difficulty, 134ff., 156ff. 

Expressing, CAVD altitude of an in- 
dividual, 365ff.; various sigmas in 
common unit, 303 ff. 

Extent of intellect (See Width) 

Face-value vs, true value of scores, 
3f., 16, 43, 53, 224ff., 497ff. 

Factors, affecting form of N* I. T. 
distribution, 276; in intellectual 
achievement, 431 

Failure-success marking, 477, 479 

Familiar tasks as tests, 439f. 

Fit, goodness of, 527ff. 

Form of curve of, altitude and age, 
463 ff,; area and age, 467 ff,; per- 
cent correct, 351ff. 

Form of distribution, by year-age, 
274f., 285ff., 294; in adult im 
group, 147, 572ff, ; in adult recruits, 
598f.; in Army Alpha, 291f,; in 
Army Beta, 291f,; in Class 3 
group, 579; in Grade 4 group, 
584f.; in Grade 5 group, 584f.; in 
im 3 group, 572ff. ; in im 6 group, 
575ff.; in Daw School group, 586ff.; 
in special class group, 580 ; of 
adults, 287ff.; of altitude in Grades 
VI to XIII, 224, 294; of an array, 
520; of college freshmen, 538ff.; of 
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QOPI intellect, 286; of Grade VI, 

Form of distribution of, Grade JX, 
521, 528ff.; Grade XII, 533ff.; 
group given composites A to D, 
142 ; intellect, 270ff., 294, 295 ; level 
in CA VD intellect, 104f . ; school 
grade population, 470, 483f.; varia- 
tions as check on estimates of diffi- 
culty, 41; variations in intelligence 
test scores, 104 

Form-boards as tests, 436f. 

Formula, for attenuation, 60, 1 16, 560, 
561 ; for coefficient of correlation, 
485 ; for correction of error in mea- 
surement of gains, 409; for correc- 
tion for restricted range, 155; for 
unreliability, 298f,, 324f., 559, 561, 
567; for rti, 177; for 302; for 
Ttj, 110; for rho, 147, 485; for 
+ for a., 110, 222, 

297; for 110; for 0„ 110, 111, 
117; for tnie o, 122; Spearman- 
Brown, 425f., 556, 567, 570 

Freui>, S., 272, 461 

Gain, as related to, errors of measure- 
ment, 409; initial ability, 409; in 
intellect with maturity, 463 IF. ; in 
intelligence test scores, 287 ffi; of 
whites vs^ negroes, 288 

GAiiTON, Francis, 1, 270 

Gates, A. I., 135 

Gaussian curve, 6, 53, 270, 521, 527, 
536 

General Electrical Information Test, 
599 

General principles and technique of 
measurement of human abilities, 
476fr. 

Generalizing vs. associative thought, 
414f. 

Goodness of ilt of, college freshmen 
curve, 641; Grade VI curve, 529; 
Grade IX curve, 527; Grade XII 
curve, 535; sumznated scores curve, 
647 


GOPI intellect, form of distribution, 
286 

Gradation of responses to tasks, 477 
Grade 6, 9, 12, variability in, 104 
Grade 9 as unit, sigma of variability 
in, 112 

Graded inventory to measure int^eet, 
412 

Greatness of intellect in terms of area 
and altitude, 387 

Growth and practice effects on Alpha 
scores, 491 • 

Haggerty, M. E., 540, 546 
Haggerty Intelligence Examination 
Delta 2, 41, 228, 274, 291, 309, 316, 
493, 510, 522, 528, 530, 633, 534; 
increase in score with age, 464ffi; 
scores in equal units, 247ff., 276, 
316, 464ff. 

Hammarberg, C., 432 
llaudwriting scales, 134 
Hearing, tests for, 4761?. 

Hereditary, ability vs. acquired abil- 
ity, 95f. ; factors producing varia- 
bility, 272 

Herring Bevision of Simou-Binet 
Tests, 403 

Higher thought processes vs. connec- 
tion-forming, 414£f., 473 
HiIjLKGas, M. B., 134 
Ho LlilNG WORTH, H. Li., 134 
HoiiiiiNGWOicTH, Li. S., 135, 433 
Homogeneity of CAVD, difficulty,'* 
lOlffi, 469; intellect, 5561?., 571 
Honesty, tests in, 476ff. 

IIUN SICKER, L. M., 400f. 

Hypotheses on increases in intellect, 
48 ff. 

Hypothesis on nature and cause of 
intellect, 412, 415 

Idea defined, 418 

I.E.K., technique, 476; Tests of Se- 
lective and Belational Thinking, 

• jL ^ 

Generalization and Organization, 96, 
101, 120, 228, 522, 528, 534; score 
in equal units, 257ffi, 316 
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lUinois Intelligenee Examination, 41^ 
493^ 522, 528; scores in equal units, 
316 

Imperfections in testing instruments, 
10, 109 

Importance of altitude of intelloett 35 
Impr/ivement in specialized abilities 
with age, 468 

Inaccuiracy of determination affecting 
ability vs. variability, 500 
Increase in number of tests with level, 
384ff. 

Increases in intellect, 48 ff. 

Individual differences, affecting corre- 
lation of altitude and width, 115, 
395ff,; in connections, 415f., 420f.; 
in original nature, 421ff. 
Inequalities in units, 306, 526 
Inferences as to nature vs. nurture, 
441ff, 

Influence, of disease and accident upon 
form of distribution of intellect, 
272; of hereditary factors upon 
variability, 272; of selection to 
simulate normality, 272 
Inherited ability vs. acquired ability, 
95f., 433ff. 

Inhibitions as connections, 417 
Initial ability vs. errors of measure- 
ment, 409 

Instantaneous change from success to 
failure at higher level, 374f. 
Instructions for rating difficulty of 
test elements, 135f. 

Integration vs. dissociation, 431 
Intel as unit, 6 

Intellect, and single tasks, 109ff., 
ISlff.; as ability to get truth, 15; 
as ability to loam, 17f.; as develop- 
ing with age, 16f.; as organizing 
ability, 19f. ; as relational thinking, 
19f.; as response to novelty, 18f.; 
CAPI«,„ 397; CAVD, 65, 96f., 99, 
lOlff., 114, 118f., 143, 159, 178, 
222f., 286, 302, 338, 374ff,, 383, 
388ff., 403ff,, 410f., 411, 458, 463, 
469ff,, 476, 556, 564, 565 ff., 574; 
CAVDI, 97, 99, 101; CAVDIO, 
154f.; CAVI>0,414; CAVl)OS,414; 


CAVDOSB,414; CEPE, 322; GOBI, 
286, 571; NIL, 222; OGAnS, 222; 
defined, 25f 412, 433, 469 ; is^onym 
for intelligence, 1 

Intellectual difficulty, 22f., 28, 62ff., 
109ff., 119, 126f., 133; as measured 
by consensus, 156ff. ; assumptions in 
measuring, 38, 59; criteria for, 
63ff. ; in Binet tests, 402 ; of CAVD 
levels, 179ff. ; of composites A, B, 
C, D vs. ratings, 142; of composites 
A to K, 335f.; of composites I to 
Q, 302ff.; of ta^s, 58; of task, zero, 
348f, 

Intellectual, level, 24, 33, 104; power 
over persons, things, ideas, 413 ; 
product defined, 25f.; ta£& defined, 
59f., 469; tasks, 26, 59ff., 413f., 
469 ; vs. non-intellectual tasks, 413f* 

Intelligence (See Intellect) 

Intelligence quotient, 433, 495f. 

Intelligence tests (See Analogies, 
Army Alpha, Army a. Army Beta, 
Binet, Brown University, Burt, 
CAVD, Dearborn, form-boards, Hag- 
gerty, Herring, I.E.B., Illinois, 
Iowa, Kelley-Trabue, mazes, Menti- 
meter. Miller, Morgan, Myers, 
National, Otis, Pintner, Pressey, 
Princeton, Smith College, Stanford 
Binet, Strickland, Terman, Thur- 
stone, Trabue, Thorndike, Yale 
Tests) 

Intercorrelation of associative and 
higher abilities, 426ff. 

Intercorrelations, for determination of 
altitude, 389, 392ff.; of 40 com- 
posites C to G, 325ff. 

Interdependence of altitude, area, 
width, 412 

Interest and capacity, 430 

Interviews as tests, 1, 15 

Inventory, as tool for measurement, 
115, 399, 469, 477f.; of intellectual 
tasks, 27, 159, 412 

Investigation needed of altitude and 
j^eed of learning, 409 

Iowa Comprehension Tests, 540, 546 
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Jastbow, J., 1 

Jad^eata of intelleetaal difScttltf, 
134ff. 

Kansas State Teachers’ College, 522 
KEtiBT, T. L,, 110, 112, 117, 156, 225 
KeUej-Trabue Completion, 510, 530, 
533 

Kuke, Zi. W., 134 

Latton-, Ih H., 522 
Level, CAVD, composites, A to D, 
66ff.; I to M, leOff.; N to Q, 76ff. 
Level, importance of, 35; intelleetaal, 
24, 33, 104 

Levels of intellect, 159£F. 

Limitation of test itemi^ conaideTation 
of, 63f. 

LimitationB of growth vs. CAYD intel- 
lect, 468 

Lincoln school, intereorrelations in 
tests, 97f., 106 

Location of zero point, 485; hy con- 
sensus, 342fl.; by experiment, 340ff. 
Lower vs. higher thought processes, 
414f.; 473 

MacPhaii,, a. H., 433, 534, 540 
Madsen, J. N., 465f., 522 
May, M. a., 99 
Maturity vs. training, 463ff. 

Mazes as tests, 21, 437f. 
McCall-Thomdike Beading Test, 7 
Meaning, of Binet’s mental age, 402; 
of Ojg, 337f. 

Measures, of difficulty from common 
reference point, 314fF., 331ff.; of 
improvability as measures of intel- 
lect, 409 

Measurement, errors of (See Errors) 
Measurement of, carpentry, 476fif.; 
compositions, 476£F.; difficulty of 
composite task, 119; discrimination 
of pitch, 476ff.; drawing, 476ff.; 
hearing, 476fl.; honesty, 476fl.; in- 
clusive vs. exdnsive traits, 480ff.; 
individual differences, 376f.; intel- 
lect by competent judgments, 39; 


intellect by inventories, 460; intel* 
leet by measurement of eonneetionsi, 
422; intellect by percent of suoeesi^ 
37; intellect by product produced, 
14; intellect by single tasks, lOOfl., 
120, 127 ; intellect, theorems foe, 
30ff.; intellect via form of distri- 
bution of arrays, 54ff.; intelleet via 
form of distribution of intelleet, 
Slff.; intellect via form of distribu- 
tion of variations, 40; intelleetaal 
difficulty, 37ff., 470; intellectuaP 
difficulty of a task, 109ff.; level, 
width, speed, 35ff,; likei* and dis- 
likes, 489f.; mechanical ability, 
476ff.; motor skill, 47 Off. ; original 
capacity, 95; popularity, 476ff.; 
products of intellect, 476ff.; sales- 
manship, 476ff.; ^eed, 32f., 482; 
spelling, 476ff.; width of intellect, 
373ff. 

Measurement, technique of, 27 

Measuring, individual OAYD intellect, 
351ff.; higher processes by novel 
teats, 438 

Mechanical intelligence testa items, 64 

Mechanical E^ill, 476ff. 

Median error of sum of ratings, 1, 
38f., 141 

Memoirs, National Academy of Sci- 
ences, 97, 291f., 405, 522, 528, 540, 
558 

Mendelian determiner, 271 

Mentaoes, 6 

Mental age, Stanford -Binet, 3, 6, 16, 
17, 97, 402, 447, 575f, 

Mental products, 12ff. 

Mentimeter Scale, 534 

Method, of deriving units of measure, 
134; of obtaining composite distri- 
butions, 521, 527, 529, 533, 535, 
d38f ., 545ff ., 549 ; proposed for con- 
structing composites, 176S.; used in 
constructing CAVD cpmposites, 178 ; 
used in*eonstracting id-composite V 
tasks, 179 

Milan, Michigan, Alpha Tests, 491 
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Miller Mental Abilit 7 Test, 534 
MiNBti, Z« F., 135, 138 
Minnesota, Opposites and Completion 
of Bciimtiojis Test, 546; Becogni- 
tion Vocabnlaiy Test, 540, 546 
Mixed Belations Teats, 9 
Mo^kof, W. S , 522 
Mokgan, J. J, B., 540 
Morgan Mental Test, 510 
Motor Skill, 476ff 
Myeus, C £ , 522 
Myers, G. C, 522 

M>ers Mental Measure, 41, 228, 493, 
510, 522, 528, 530, 533; scores m 
equal units, 254, 316 

National Academy of Sciences, Mem 
oxrs, 97, 291f , 405, 522, 528, 540, 
55S 

National Intelligence Test, 1, 2, 15, IS, 
41, 52, 99, 107, 222, 227, 274, 291, 
40 i, 404, 408, 464, 475, 493, 522, 528, 
530, 533, 576f ; distributions of, in 
equal units, 276t ; selection of tasks 
for, 409; sroies in equal units, 
239fiF , 307, 309, 316 
National Bosearch Council, 9 
Natural zeros, 487f 
Nature of mtelhgoneo, 412fP. 

Natuic nurture, 95±., 43G, as mea 
suied by tasks, 441f ; as related to 
altitude and width, 458 
Need, for knowledge of number of 
tasks at each level, 379; for wide 
inngc and large population to find 
vai lability at levels of ability, 499; 
for zeio as point of roferenee, 339 
Negative, acceleration, of level with 
age, 467, 475; and positive factors 
in the con«ititutTon of intellect, 50; 
skewness in special class gioup, 580 
Neural mechanisms vs, thought proc- 
esses, 415 
Noble, E, E., 539 
Normal distribution, 55, 115 
Normal piobabhity curve, 6f., 53, 270; 
as distribution of Grade IX Intel 
lect, 521 


Normal probability equation, 6, 270^ 
470, 521 

Normal probability surface, 470, 527, 
585 

Normality of distribution, in adults^ 
274fE.; in age groups, 287ff., 293; 
in gioup, arguing skewness in gen- 
eial, 224, 294, 296; in grade groups, 
224, 294, 296 

NoRS WORTHY, N., 9 

Notation for width of intellect, 337 

Novel tasks as tests, 18f., 457f. 

Number of connections as related to 
ontogeny and phylogcny of intellect, 
432 

Number of tasks at vaiious levels, 457, 
472 

Numerical data as tests, 20f. 

Officers^ ratings and Army Alpha 
Scores, 405 

One idea defined, 418f. 

Ontogeny of intellect cf. numbei of 
connections, 432 

Order, of intellectual difficulty, 27, 62, 
65 

Original, capacity and CAVD intellect, 
435, capacity to bo correct as due 
to ntunber of conneetious, 416 ; 
nature, individual differences in, 
421; f?9. acquiied ability, 4S3 

OSBUKN, W. J , 528, 534 

Oj is, a. S , 522, 528 

Otis, Advanced Examination, 14f., 41, 
52, 228, 274, 291, 403, 404, 464, 493, 
522, 528, 530, 533, 534, 577; scores 
in equal units, 245S , 27Gffi., 307, 
309, 316; Intermediate Test, 599; 
Primary Examination, 41; Self- 
Admmistering Gioup Test, 96f , 99, 
106, 405ff. 

Over estimation in consensus ratings, 
143, 157 

Over lapping as measure of correspon- 
dence between single task and intel- 
lect, 131f 
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Parabola as curve of altitude and age, 
460 

Partial correlation, between speed, 
altitude, errors, 401 ; affected hy 
error in measures, 454f.; in Bart 
tests, 4513.; with intellect, of ele- 
ments in a composite, 128f. 
Patekson, D. G., 540, 546 
Peabson, K., 527 

Pearson, correlation cf. Sheppard, 97, 
99, 101, 298, 323f,, 391f-, 557, 559, 
565ff-, 570; formula for correction 
for range, 556 ' 

Percent successful as related to diffi- 
culty, 37, 351fP. 

Phylogeny of intellect ef. number of 
connections, 432 

Physiological, cause of intellect, 412, 
420f. ; facts of intellect, 473 
Pictorial data as tests, 20 if. 

PiNTNER, R., 135, 522, 528 
Pintner Non -Language Examination, 
222, 228, 522, 528; scores in equal 
units, 254ff., 316 

Positive and negative factors in the 
constitution of intellect, 50 
Practice and growth clTects in Alpha 
scores, 491 

Pkessey, 8. L., 522, 528 

Pressey Mental Survey, 322, 528; 

scores in equal units, 316 
Princeton. Intelligence Examination, 
540, 546 

Principles, for separating original 
from acquired abilities, 436; of in- 
tellectual difficulty, 22, 24, 3 Of. ; of 
measurement of human abilities, 
476ff. 

Probability curve, 6f,, 53, 270, 470, 
521, 527, 685 

Probable error, of determination vs. 
unreliability, 520; of medians in 
ratings, 144f.; of sum of ratings, 
138f., 141 

Product of intellect defined, 26 
Program, for measuring tasks of small 
intellectual ^fficulty, 341; of ex- 


periments to locate zero intellect, 
3396!. 

Proportional vs. absolute counts bf 
tasks, 3S3f. 

Psychological theory criteria for test 
items, 63 

• 

Qualitative differenecs in intellects, 
421 

Quality of intellect vs. quantity of 
connections, 415f!. 

Quality of response, as criterion of 
groe of intellect, 416; vs. success- 
failure marking, 477, 479 

Quantity hypothesis of intellect, 415fl. 

Quickness of intellect (See Speed) 

Range of intellect (See Width) 

Range, restricted, correlations found 
in, 297 

Bank order of difficulty for teat ele- 
ments, 38 

Ratings for difficulty of test ele- 
ments, 

Beading composites, construction of, 
193ff. 

Reasoning tests, Burt’s, 447flr. 

Reference point in CAVD, 295, 314ff., 
331 ir. 

Regression, equation in Burt results, 
447, 453; line in ability vs. varia- 
bility, 504 

Relation, b^'tween altitude, width, 
area, speed, 3881?., 400f.; between 
ability and variability, 43, 104, 
497 ff.; between Binet M.A. and 
CAVT> level, 402 ; between tasks and 
heredity, age, sex, 484f.; of CAVD 
to other intelligence examinations, 
96ff 105 ; of error of measuromeut 
to initial ability and gain, 408f. 

Relational thinking as intelligence, 19 

Relative, magnitudes of levels A and 
Q, 33 7f.; variability of grade popu- 
latiozv, 306; vs. ab^late counts of 
tasks, 383f. 

Reliability, of judgments, 140 ; of 
ratings, 138f.; of score determina- 
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tion, 140, 410; of tort materiBl, 
lS9f., 324fl., 567 

Bemoteneas, effect of, on oorrelationa, 
567, 562 

Eepeated, Alpha testa, 401; Oourtis 
teats, 403; Woody testa, 493 
Bec[^icted range, correlatiozifl found 
in, 307 

Bobiisbon, £. E., 135, 138 
Booebs, a. TU, 438, S4D 
Bogbbs, D. C., 540, 546 
Bogere Syllogism Test, 436 
BowsnL, D. C., 403, 463 
Buoh, G. M., 534, 540, 546 
Buoeb, G. J., 135 

Bole for estimating OAYD altitude of 
individual, 360 

Salesmanship, 467 ff. 

Sampling, error of, lO0f., 112, 114, 
117f.; suficiont, for correlation of 
one level with entire series, 565 
Satiafyers and annoyers, 489f. 

Scale, CAVD, 204ff.; OAVD Levels 
A-D, 66ff.; CAVD Levels I-M, 
leoff.; CAVD Levels N-Q, 76ff. 
Beales for composition, designs, draw- 
ing, handwriting, 134 
Scores in teats in equal tinits, 43ff., 
105, 2S4ff,, 306ff., 403, 475, 497ff,, 
576, 584, 686, 592 

Scoring, by suecesB-failure es. graded 
^ credit, 477, 479; products of intd- 
lect, 22f., 29 

Selection of word-knowledge tasks, 183 
Selective associative thought, 414f. 
Self-correlation, of altitude measure, 
389ff.; of Army Alpha, 318; of 
composites, 297, 825; of Haggerty 
test, 318; of I.E.B. tests, 101; of 
Otia teste, 99 ; of percent a at levels, 
391ff.; of ratings, 138; of sub- 
series in CAVD, 107; of task, 60; 
of Terman ^roup Test, dlTf.; of 
Thorndike Intelligence Examination 
H.S.G., 97 ; of various combined 
teats, 318 


Sensori-motor conneetioiis and inteiQi** 
gence, 418, 47617. 

Sheppard correlation eoefficieiit e/. 
Pearson’s, 97, 99, 101, 298, 391, 
392, 567, 550, 665ff., 670 

Shifting curves of percent correct, 
353ff. 

Shrinking of units, 44 

Sigma, as unit of measure, 43, 50, 53, 
64, 57, 109ff., 225; of army in Han- 
ford M.A., 97; of CAVD altitude 
score, 411 ; of errors, 126 ; of group 
as unit, 295ff., 303, 323f.; of imbe- 
cile group, 154f.; of sum of ratings, 
138f. 

Sigmas, of groups in common unit, 
303ff.; of vaxiouB tests in common 
unit, 317 

Significance of scores in stock exami- 
nations, 403f. 

SmPSON, B. B., 9 

Single tasks and intellect, 109ff., 131ff. 

Skewness, 271; cf, differential gain in 
score, 290 ; in adult distribution, 
287ff., 293; in special class group, 
580 

Smith College Examination, 540, 546 

Smith, B. M., 540 

Smoothiag for chance error, 299 

Social intelligence items as data for 
tests, 64 

Space-forms as data for tests, 20f. 

Spearman, C., 9, 19, llOff., 391, 394, 
559 

Spearman-Brown correction for sdf- 
correlation, 298, 323, 394, 425f., 556, 
659, 669, 570 

Specialization, in CAVD lev^s, 397; 
in intellect within levels, 390f. 

Speed, as measure of intellect, 24f., 
32, 104; factor in stock intdligence 
examination, 404f . 

Spelman Memorial Fund, 442 

Spelling, tests in, 476ff. 

Stanford-Binet, 1, 3, 8, 15, 22, 38, 41, 
55, 96, 97, 99, 106f., 433, 439, 475, 
493, 572f., 576; q^keed and altitude, 
40Sff. 
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Btatu of uenroneB cf» number of con* 
neetioiui^ 432 
SVBCSKBy Ii« M., 405 
Stskquist^ J. L.^ 9^ 530 
Stren£fth of oonnections, 480 
Strickland Test, 534 
BtbicsxiAND, V. Li., 534 
Strong, K., 184 
Sub-aeries, of CAVD tasks, 66fr., 
160ff«; of taaika in levels, 484 
Substitution Test, 222; as measure of 
ability to learn, 408f* 
Succeas-failure vs. graded credit 
scoring, 477, 479 

Sufficient sampling for correlation of 
one CAYD level witb entire aeries, 
565 

Summary of eonclusiona, 469ff. 
Swelling and shrinking of units, 44 
Symmetry in distribution of intellect, 
270f. 

Tape, H. A., 491 

Tasks, intellectual, 26, 59£P., 413f., 
469; for location of zero by con- 
sensus, to measure altitude, 

183 

Technique of measurement of intellect, 
27 

Techniques of measurement of human 
abilities, 476ff. 

Terman Group Test, 42, 52, 96f., 99, 
106f., 228, 494, 534, 577; scores in 
equal units, 250ff., 309, 816; speed 
and altitude, 405ffi 
Tebmak, Li. M., 8, 9, 16, 183, 433f., 
576 

Terms expressive of altitude and area 
of intellect concept, 379 
Tests (See Analogies, Army Alpha, 
Army a. Army Beta, Binet, Briggs, 
Brown University, Buckingham, 
Burt, GAVD, Courtis, Dearborn, 
Drawing, Bbbinghaus, Form-boards, 
General Electrical, Haggerty, Hand- 
writing, Herring, I.E.B., 

Iowa, Kelley-Trabue, Mazes, Mcmti- 
meter, Miller) Mixed BelaUons, Mor- 


gan, Myers, National, Otis, Piisfcner, 
Pressey, Princeton, Bogers, Smith 
OoUege, Stanford-Binet, Stexkqvdflt, 
Btriddud, Substitution, Terman, 
Thorndike, Thomdike-MeCall,Tkur8- 
tone, Trabue, YanWagenen, Word 
Knowledge, Woody, Yale) ^ 
Tests, indicative of original eapaeity 
vs. nurture, 44217.; more or 1^ sus- 
ceptible to training, 440f , ; types of, 
9, 20f. 

Tetrachoric r, 390. 

Theorems on measurement of intellect, 
30ff. 

Thomson, G. H., 135, 138, 409 
Thorndike, E. D., 112, 134f 188, 288, 
408, 521 

Thorndike Intelligence Examination 
for High School Graduates, 42f., 
96f., 107, 130, 494, 497, 500, 50317., 
515, 518, 534, 540, 546, 586 
Thomdike-McGall Test of Beading, 7 
Thorndike Yisual Yocabulary Test, 510 
Thought processes vs. neural mechan- 
isms, 415 

Thubstone, It. It., 79, 540. 
Thurstono Psychological Examination, 
540 

Tidton, J. W., 35, 422f., 430, 443 
Trabue Completion Tests, 119, 528, 
528; scores in equal units, 316 
Trabue Mentimeter, 228 
Trabue, M. B., 522, 528 
Training, and CAYD intellect, 435f.;^ 
on test items, 437; vs. maturity, 
463ffi 

Transformation of scores into equal 
units, 224ffi., 306ff. 

Transmutation of arbitrary scores to 
make variabilities equal, 520 
True value vs. face-value scores, 3f., 
16, 43, 53, 224ff., 497ff. 

Truth as criterion for value, 14f. 


Under-^timation in eonsensus rating^ 
157 ’ 


Unequal units, 306ft. 

Unitary oauae of intelleet, 412 
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Units, equal, in intelligetiee teats, 
43ff., 105, 224ff., 403, 476, 497ff., 
576, 584, 586, 592; inequalities in, 
306, 526; of measure, method of de- 
riTing, 134; swelling and shrinking 
of, 484 

Uni^ of intellect, 63 ; OAVD, 399 

Unreliability, of jadgznexita,U38f.; of 
rativgs, 140; of score determination, 
122, 410; of test material, 129f. 

Use, of an ays to measure difficulty, 
<i40; of tests more and less sus- 
ceptible to training, 440f» 

Validity of consensus of judges in 
ratings, 141 

Valuation, of mental products, 12ff.; 
of responses in measuroment, 477 

VanWagenen History Scales, 11^ 

Variable errors in consensus ratings. 
142 

Vnriability, for various grade popula- 
tions, 311; for various tests, 308ff. ; 
in an array, 56; in correlation for 
single tasks and intellect, 122; in 
expcils’ opinion of easy tasks, 349; 
in face-value va. equal unit scores, 
308fF.; in grade 6 w. college stu- 
dents, 132; iu grade 9 va. age 20, 
562; in Grades VI to XIII, 306; in 
Grades VI, IX, XII, 104; in low 
level groups, 325ff.; in sigmas, 136; 
of group as afPeeting correlations, 
126f.; of group as unit of measure, 
109, 493; of individual, 40f., 43ff., 
56, 61, 104, 470f,; of individual 
measured from correlation array, 
502; of literates in draft, 562; va. 
ability, 497ff. 

Variation, chance, eliminated, 590ff*; 
in the I.Q., 495f . ; in judgments, 
136 ; in magnitude of a unitary 


cause as measure of intellect, 412 
Variety in test forms, 21f, 

Various sigmas in a common unit, 
303f., 328ff* 

Verbal data as intelligence tests, 20f» 
Verification of quantity hypothesis of 
intellect, 422fl. 

Vincent, L. M., 122, 129fr., 135, 138 
Vocabulary levels, ISOfl. 

Volume or area of intellect, 378ff. 

Watson, J. B., 272 
Whipple, G. M., 9, 433flP., 622 
Width of intellect, 25, 31f., 33, 104, 
469; and nature vs. nurture, 458; 
and stock examination score, 403f.; 
as related to altitude, 473; as re- 
lated to number of tasks at a level, 
379; in sample in entire series, 
374; notation for, 377; va. altitude, 
387fP. 

Wood, B. D., 135, 225 
Woodworth, R. B,, 9 
Woody Tests, 493 
WOODYARD, E., 135, 138, 410 
Word knowledge, composites, 186ff.; 
levels, ISOff. 

Yale Examination, 541 
Yebees, B. M., 9 

Zero, deviations from median, 494; 
for Binet scale, 403; in discrimina- 
tion of pitch, 486; influence^ of re- 
moteness on correlation, 556; intel- 
lectual difficulty task, per- 

cent correct, 378; point, 4, 27, 36, 
294, 336f., 339ffi, 441, 471f., 482, 
4S5f.; points, natural and conven- 
tional, 487f.; training upon test 
items, 437 




